TY - JOUR
AU - Tian, Feng
AB - Abstract Pointing and gesturing with the dominant thumb (DT) and the non-dominant thumb (NT) are two common tasks for bimanual tablet gripping interaction. Understanding the differences between DT and NT is important to pointing and gesturing based interface design on tablets, but is overlooked by previous studies. We therefore conducted two experiments. In the first experiment, participants carried out pointing tasks with DT and NT, respectively, on a tablet. We found DT and NT input differed in pointing time (DT had average shorter time than NT) and required target sizes for fast and accurate pointing (7.7 mm and 9.4 mm in diameter for DT and NT). On the other hand, DT and NT were alike in pointing accuracy. They performed about the same for target size larger than 9.6 mm and distance shorter than 21 mm. Both DT and NT pointing can be modeled by Fitts’ law. In the second experiment, participants performed gesturing tasks with DT and NT respectively on a tablet. The collected data were analyzed using a set of gesture features. Results showed DT and NT were different in features such as articulation time, size ratio and indicative angle difference, but similar in features like aperture, axial symmetry and shape distance. The differences of time and accuracy between DT and NT depended on gesture complexity but not gesture sizes. We discuss these findings with implications for future bimanual thumb interaction design and research. RESEARCH HIGHLIGHTS A first study was conducted to compare the dominant and the non-dominant thumbs with bimanual tablet gripping interaction. Results revealed many differences and similarities between the two thumb types for gesturing and pointing tasks. While the dominant thumb generally outperformed the non-dominant thumb, the two thumbs performed similarly in cases such as pointing to large targets and in gesture features like shape distance. Our study contributes to bimanual table interface design both theoretically and practically. Results revealed not only enrich Guard’s theory (Guiard, 1987) in the context of bimanual tablet gripping interaction, but also have many design implications. 1. INTRODUCTION Tablet computers have arguably become a popular mobile device since the advent of the Apple iPad in 2010. Tablets usually have larger screen estate than mobile phones, thus gaining wider usage in many day-to-day activities like viewing and editing documents and watching videos (Müller et al., 2012). A common way to use a tablet is holding it with both hands and inputting with DT and NT (Odell and Chandrasekaran, 2012; Wolf, 2015) (Fig. 1). Such a way has two intuitive benefits: the grip is stable and interaction tasks can be assigned to both thumbs to alleviate hand fatigue. Prior studies have altered many existing interfaces to enable them to accommodate the bimanual gripping interaction style for the purpose of making use of the benefits of such two-handed input, for example, gesture keyboards (Bi et al., 2012), marking menus (Kin et al., 2011) and QWERTY keyboards (Oulasvirta et al., 2013). Figure 1. View largeDownload slide Bimanual tablet gripping interaction. Figure 1. View largeDownload slide Bimanual tablet gripping interaction. For tablet interaction, bimanual thumb pointing and gesturing are two fundamental tasks which are performed with DT and NT alternately or simultaneously (Wolf, 2015). This raises many basic questions. For example, what are the differences and similarities between DT and NT input? Are design recommendations for DT use applicable to NT use or vice versa? To what extend could interface design ignore their differences? According to previous studies (e.g. Guiard, 1987) and our daily experience, the dominant hand has higher degrees of diversity than the non-dominant one, so the former hand type is usually assigned to precise actions and the latter hand type is restricted to simpler actions or used as a frame of reference. In addition, studies also show that the dominant and non-dominant hands have their own strength and weakness, for example the dominant hand is good for selecting small targets and the non-dominant hand is well suited for scrolling tasks (Kabbash et al., 1993). Given the close correlation between mobility of the thumb and hand function, DT and NT should own many different and similar characteristics in pointing and gesturing behaviors. In order to address the raised questions, we should understand these characteristics of DT and NT input and provide design suggestions by carefully considering their differences and similarities. Taking bimanual pointing as an example. Determining the required target size for fast and accurate pointing is important to user interface design. If such a size for DT pointing differs from NT pointing, we need to suggest designers adopt an exclusive size for each thumb use or a size large enough for both thumbs. Taking bimanual gesture input as another example. If gesture features are quite similar for both DT and NT drawn gestures, certain user interfaces can be designed to be equally DT or NT friendly within these features. While the need to examine the differences and similarities between DT and NT input is significant, to our knowledge, little research has been conducted on such topic. Existing studies have investigated unimanual thumb interaction on mobile touchscreen phones (one-handed tapping Parhi et al., 2006; Perry and Hourcade, 2008; Trudeau et al., 2012; Wobbrock et al., 2008 and one-handed gesturing Bragdon et al., 2011; Tu et al., 2015), front- and back-of-tablet interaction with grasping hands (Wolf and Henze 2014; Wolf et al., 2014; Wolf, 2015), but none of them were focused on the comparison between DT and NT for pointing and gesturing tasks with bimanual tablet gripping interaction. We therefore ran two experiments to evaluate bimanual thumb pointing and gesturing on a touchscreen tablet. We aimed to look into the differences and similarities between DT and NT input by examining a set of selected measures. We then discussed the results and concluded our study with design implications. 2. RELATED WORK Two-handed interaction has gained much research attention in fields such as human–computer interaction, ergonomics, psychology and physiology, as bimanual manipulation can afford manual and cognitive advantages in a vast category of manipulations (Annett et al., 1979; Buxton and Myers, 1986; Guiard, 1987; Leganchuk et al., 1998; Matthieu et al., 2016; Todor and Doane, 1978). A detailed review can be found (Buxton et al., 2017; Jones and Lederman, 2006). It is widely accepted that non-dominant and dominant hand manipulations are asymmetric due to their differences in degrees of diversity, and the non-dominant hand usually serves a reference function while manipulation proper seems to be carried out by the dominant hand (Buxton et al., 2017; Guiard, 1987). For example, the non-dominant hand can be used to modify interaction by the dominant hand on tablet interaction (Wagner et al., 2012). However, for some HCI tasks like pointing and dragging tasks, previous studies also indicate that the two hands are complementary and each having its own strength and weakness (Kabbash et al., 1993). Two-handed interaction generally falls into two categories (Balakrishnan and Hinckley, 2000): integrated bimanual interaction where left and right hands perform highly integrated actions like two-handed zooming operation, and separated bimanual interaction where two hands execute non-integrated tasks like typing on a QWERTY keyboard with two hands. Our study focuses on the latter category, which has two main cases: pointing based and gesture oriented interactions. We review related work as follows. 2.1. Bimanual thumb pointing Pointing tasks are ubiquitous for bimanual tablet gripping interaction. Wolf and Henze (Wolf and Henze, 2014) conducted a study to compare four pointing techniques (direct pointing, indirect pointing on a virtual touchpad, an inverse cursor, and a miniature interaction area) on tablets. Direct pointing was the fast technique but was limited by its interaction area. The miniaturized interaction area technique was the second fast technique and can serve as a promising alternative to direct pointing. Wolf et al. (2015) also investigated biomechanics of front- and back-of-tablet pointing with grasping hands, but they did not compare the performance between NT and DT. Two-handed typing on a QWERTY keyboard is a typical example of bimanual thumb pointing. Research has investigated touch behavior on soft QWERTY keyboards when used with two thumbs as well as an index finger and one thumb (Azenkot and Zhai, 2012). While no statistical results were reported explicitly, the data in the study (Figures 9 and 10 in Azenkot and Zhai, 2012) indicate right thumb tapping differed from left thumb tapping in touch point distributions, vertical and horizontal touch point offsets. To improve split keyboard design for text entry with two thumbs on mobile touchscreen devices, Oulasvirta et al. (2013) looked into a number of related factors including button sizes, keyboard shapes and positions, letter-to-key assignment and online error correction. However, their study did not focus on differences between left and right thumb pointing. To aid mini-QWERTY keyboard design, two-handed thumb typing models have been proposed to predict expert text entry speed on such keyboards (Clarkson et al., 2007). Trudeau et al. (2013) investigated the effects of tablet keyboard configuration on thumb typing in a two-handed grip. The split keyboard had a lower typing speed but required less amount of reaching by the thumb than the standard layout. Additionally, typing discomfort was the lowest with keyboards in the middle location of the screen. 2.2. Bimanual thumb gesturing Gesturing with thumbs is common for bimanual tablet gripping interaction. To extending the marking menu paradigm from one finger to multiple fingers, Kin et al. (2011) proposed two types of bimanual multistroke marking menus. One is two-handed ordered, where users alternate strokes between hands. The other is two-handed simultaneous, where users draw strokes with both hands simultaneously. Results showed the two-handed simultaneous marking menu was faster than the two-handed ordered one by 10–15%, while the latter was similar in performance to the unimanual marking menu. Bi et al. (2012) created a variant of unimanual gesture keyboards to make them compatible with two thumb use. Although the modified gesture keyboard had slightly inferior speed efficiency over unimanual gesture keyboard, it was preferred by users as it raised comfort levels and reduced physical demands. Wolf et al. (2014) investigated how users drew four simple gestures (tap, press, drag and swipe) on front and back of tablets devices with grasping hands, but their study was not focused on the comparison between NT and DT. 2.3. Models for pointing and gesturing tasks Our study adopted pointing and gesturing tasks to evaluate thumb performance. We review models related to the two tasks and used them in our analysis. Fitts’ Law is often used as a model for pointing operations in user interfaces to predict movement time from a start target to an end target (Fitts, 1954). According to the law, movement time is determined by the distance between the start and end targets, and the end target width in the direction of travel. Although there are many variations on the formula, a common form of the law for movement time ( MT) predictions is MT=a+blog2AW+1, where A means the distance between the centers of the start target and the end targets, W represents the width of the end target in the direction of travel, and a and b are constants reflecting the efficiency of the pointing system. The logarithmic term log2AW+1 denotes a pointing task’s ‘index of difficulty’ ( ID), measured in bits. An experimental pointing task typically requires tapping the start target first then tapping the end target as quickly and accurately as possible. A detailed review can be found in MacKenzie (1992). Freehand gesturing is an inherently complicated motor control behavior. One aspect of gesture design is to measure and characterize gesture complexity, which has attached much attention research. Isokoski (2001) proposed a model for stroke gestures that used the number of approximating straight line segments in a gesture as a predictor of complexity correlating to production time. A more formal model, the CLC model computes a gesture’s production time based on sub models of curve, line, and corner production (Cao and Zhai, 2007). We used the CLC model as a verification method in the design of our experiment. From the literature review, we can learn that two-handed thumb pointing and gesturing are two important interaction modalities in user interfaces. However, little research paid attention to the differences between DT and NT interaction. Given the significant need to examine the differences presented in Section 1, our study is aimed at shedding light in this area. 3. EXPERIMENT ONE: INVESTIGATING DOMINANT VS. NON-DOMINANT THUMB POINTING 3.1. Questions for investigation The purpose of this experiment was to investigate the performance between DT and NT pointing. While we were generally interested in their differences and similarities, the following questions and expectations were the main focuses of this experiment. Q1 Would the differences and similarities between DT and NT depend on target sizes and distances? The dominant hand has greater dexterity than the non-dominant hand, hence DT should have higher average speed and accuracy when clicking small and far targets. However, such differences may be diminished for large and near targets. Q2 What is the required target size for fast and accurate pointing with DT and NT respectively on tablets? Determining such sizes plays an important role for interface design. Previous studies have examined optimal target sizes for interaction with a stylus on a Wacom tablet-cum-display (Ren and Moriya, 2000), index fingers on a desktop-sized display (Colle and Hiszem, 2004), and one-handed thumb on a touchscreen handheld (Parhi et al., 2006), but none have paid attention to bimanual thumb use on a tablet. We anticipated the required target size for each thumb would be different, as DT is more dexterous than NT. 3.2. Experiment apparatus We conducted the experiment on a Google Nexus 10 multitouch tablet. The screen was 10.1 inches in size with a resolution of 2560×1600 pixels (23.4 cm in width and 14.6 cm in height). The bezels of the screen gripped by both hands were 2.1 cm in width. The weight was around 600 g. 3.3. Participants Twelve unimpaired right-handed participants (6 male) with mean ( ±SD) age of 23.0±3.4yrs and thumb length of 5.7±0.7cm took part in the experiment. All had experience in using touchscreen smartphones with 4 years on average. Ten of them have used tablet devices for 2 years on average. 3.4. Experiment design The experiment was a within-subject repeated measures design. The independent variables and dependent variables are as follows. 3.4.1. Independent variables The independent variables were thumb types (DT and NT), target sizes and target distances. We used circular targets with five levels of diameters: 3.8, 5.8, 7.7, 9.6 and 11.5 mm. Target sizes were selected to reflect the sizes of common UI elements on tablets. For example, a file icon is ~4 mm wide. An icon on the home screen is around 11 mm wide. Target distance was calculated as the length between the center of the start button (a blue circle with 20 mm diameter) to the center of the target (a red circle) (left image of Fig. 2). It had three levels (21, 39 and 58 mm). When clicking near targets (target distance=21mm), the angle between the metacarpal and the proximal phalanx of the thumb is around 90°, as shown in Fig. 3a. The angle becomes approximate 180° if the user fully stretches the thumb to reach far targets (target distance=58mm) (Fig. 3c). If the angle is between 90° and 180° (Fig. 3b), the pointing target is in the middle distance (target distance=39mm). The start button was 74 mm under the upper bezel and 7 mm from the bezel gripped by the input hand (right image of Fig. 2). For each target distance, five targets (red circles) were evenly located on a semicircle with the target distance as its radius and the start button as its center. In total, there were 15 target positions for each thumb input (right image of Fig. 2). Figure 2. View largeDownload slide The left image shows the experiment interface. The blue circle is the start button. The red circle is a target. The right image shows 15 target positions in near, middle and far distances for left thumb use. Note the targets for left and right thumb input were symmetrical about the conjugate axis of the device. Figure 2. View largeDownload slide The left image shows the experiment interface. The blue circle is the start button. The red circle is a target. The right image shows 15 target positions in near, middle and far distances for left thumb use. Note the targets for left and right thumb input were symmetrical about the conjugate axis of the device. Figure 3. View largeDownload slide Thumb postures for bimanual tablet gripping interaction. The angle between the metacarpal and the proximal phalanx of the thumb when performing pointing tasks: (a) ≈90°; (b) >90°and<180° and (c) ≈180°. Figure 3. View largeDownload slide Thumb postures for bimanual tablet gripping interaction. The angle between the metacarpal and the proximal phalanx of the thumb when performing pointing tasks: (a) ≈90°; (b) >90°and<180° and (c) ≈180°. 3.4.2. Dependent variables The dependent variables were pointing time and error rates. Pointing time was computed as the period from clicking the start key to clicking the target. A clicking error was committed if the touch point was outside the target. Error rate was defined as the ratio of number of errors to the number of trials. 3.5. Task and procedure The experiment consisted of a practice phase and a test phase. In the practice phase, participants were first given instructions on how to do the experiment task. They were asked to sit in a chair to perform the task with two hands holding the sides of the tablet comfortably in the landscape view. The start of a trail was shown as the left image of Fig. 2. The participants were asked to click the blue start button first, then click the red target as quickly and accurately as possible. Next trial started after the participants clicked the target. Participants practiced 75 trials (5 target sizes × 3 target distances × 5 target positions in each distance) by the left and right thumbs respectively. The order of trials was randomized. In the test phase, each participant completed four blocks for each thumb use. As practice, each block had 75 randomly ordered trials. The order of using left or right thumb was counterbalanced across participants. The participants on average took 25 minutes to complete the experiment. In summary, the experiment consisted of (excluding practice trials) 12 participants × 75 trials × 2 thumb types × 4 blocks=7200 trials. 3.6. Results and analysis We first checked the learning effect on pointing time and errors over the four blocks to see if the data collected had reached a level of stability. Repeated measures ANOVA and post hoc comparisons with the Bonferroni adjustment were used in the data analysis. No significant difference was found in the four test blocks for both right ( P>0.05) and left ( P>0.05) thumb use. One reason may be that pointing was a relatively simple task, so after a practice block, participants were able to reach a level of steady performance shortly at the beginning of the test phase. The data collected in the test phase were used in the following analysis. As our participants were all right-handed, we replaced the terms of left thumb and right thumb with NT and DT respectively to describe more general conclusions. 3.6.1. Pointing time We began by looking at mean pointing time by thumb types (NT vs. DT) averaged across other independent variables (target sizes and distances) to provide an overview of NT and DT tapping performance. Thumb types had a significant main effect on pointing time ( F1,11=17.39,P<0.001) (Fig. 4). The mean time was 566 and 510 ms for NT and DT, respectively. DT use resulted in an overall shorter time than NT use. Participants were more skillful with their dominant hands when performing pointing tasks. As anticipated, there were significant main effects for target sizes ( F4,44=29.41,P<0.001) and distances ( F2,22=55.42,P<0.001). Generally, pointing time increased as target sizes decreased and target distances increased. Figure 4. View largeDownload slide Mean pointing time (ms) for NT and DT use in five target sizes (left) and three target distances (right). Error bars represent 0.95 confidence interval. Figure 4. View largeDownload slide Mean pointing time (ms) for NT and DT use in five target sizes (left) and three target distances (right). Error bars represent 0.95 confidence interval. As illustrated in Fig. 4, significant interaction effects were found for thumb types × target sizes ( F4,44=32.34,P<0.05). For target sizes smaller than 7.7 mm, DT resulted in shorter average time than NT (all P<0.01). However, for target sizes larger than 9.6 mm, DT and NT had comparable average time (for 9.6 mm size, P=0.12; for 11.5 mm size, P=0.21). In addition, there was a significant interaction effect was found for thumb types × target distances ( F2,22=41.37,P<0.01). When target distance was 21 mm, DT and NT had similar average time ( P=0.11). However, DT led to shorter time than NT if target distances were 39 and 58 mm (both P<0.01). The mean value and SD for the five sizes and three distances are listed in Table 1. Table 1. Mean pointing time and SD for NT and DT use in five target sizes and three target distances. Target size (mm) Target distance (mm) 3.8 5.8 7.7 9.6 11.5 21 39 58 NT Mean (ms) 631 609 581 523 486 409 580 709 SD (ms) 74 69 62 61 58 57 68 75 DT Mean (ms) 570 542 505 473 460 389 517 624 SD (ms) 73 69 64 59 54 56 69 75 Target size (mm) Target distance (mm) 3.8 5.8 7.7 9.6 11.5 21 39 58 NT Mean (ms) 631 609 581 523 486 409 580 709 SD (ms) 74 69 62 61 58 57 68 75 DT Mean (ms) 570 542 505 473 460 389 517 624 SD (ms) 73 69 64 59 54 56 69 75 View Large Table 1. Mean pointing time and SD for NT and DT use in five target sizes and three target distances. Target size (mm) Target distance (mm) 3.8 5.8 7.7 9.6 11.5 21 39 58 NT Mean (ms) 631 609 581 523 486 409 580 709 SD (ms) 74 69 62 61 58 57 68 75 DT Mean (ms) 570 542 505 473 460 389 517 624 SD (ms) 73 69 64 59 54 56 69 75 Target size (mm) Target distance (mm) 3.8 5.8 7.7 9.6 11.5 21 39 58 NT Mean (ms) 631 609 581 523 486 409 580 709 SD (ms) 74 69 62 61 58 57 68 75 DT Mean (ms) 570 542 505 473 460 389 517 624 SD (ms) 73 69 64 59 54 56 69 75 View Large As there were apparent differences between NT and DT input in time, we continued to analyze time performance with regard to target sizes for each thumb input. Target sizes had a significant main effect on pointing time for NT use ( F4,44=28.46,P<0.001) and for DT use ( F4,44=23.55,P<0.001). The mean value and SD for the five sizes are listed in Table 1. As target size increased, participants were able to tap them faster. For the NT condition, apart from between 9.6 and 11.5 mm, pairwise differences between other sizes were significant ( P<0.001). And for the DT condition, there were significant differences between all sizes ( P<0.01) except 7.7, 9.6 and 11.5 mm. 3.6.2. Fitts’ law analysis Fitts’ law predicts the time required to acquire targets in user interfaces. A common form of the law for movement time ( MT) predictions is MT=a+blog2AW+1 (MacKenzie, 1992), where A means the distance between the centers of the start key and the target, W represents the width of the target in the direction of travel, and a and b are constants reflecting the efficiency of the pointing system. The logarithmic term log2AW+1 denotes a pointing task’s ‘index of difficulty’, ID, measured in bits. The IDs in our experiment were from 1.5 to 4.0. We analyzed linear regression of pointing time by ID. As illustrated in Fig. 5, for each thumb pointing, the regression showed high correlations with Fitts’ law: all R2 were greater than 0.95 (i.e. 95% of the variance in pointing time can be explained by the Fitts’ model). The Fitts’ model also well explains the decrease in tap time with the increase in target sizes, and hence decrease in task difficulty. The result is consistent with studies (Parhi et al., 2006; Perry and Hourcade, 2008) which showed a strong model fit for one-handed thumb tapping. Figure 5. View largeDownload slide Relationship between pointing time and index of difficulty (ID). Figure 5. View largeDownload slide Relationship between pointing time and index of difficulty (ID). 3.6.3. Error rates Thumb types had no significant main effects on error rates ( F1,11=3.64,P=0.15), indicating NT and DT had a comparable accuracy. The mean error rate was 4.4% and 3.9% for NT and DT use, respectively. Target distances had no significant main effects on error rates ( F2,22=0.67,P=0.51), but target sizes had ( F4,44=35.87,P<0.01). Larger sizes tended to have lower error rates. There was no interaction effect on error rates for thumb types × target distances ( F2,22=3.31,P=0.64). NT and DT had similar accuracy for the three target distances. The mean value and SD for the three distances can be found in Table 2. There was a significant interaction effect for thumb types × target sizes ( F4,44=0.51,P<0.05). As shown in Fig. 6, the accuracy differences between NT and DT decreased when the target size increased. Table 2 lists the mean value and SD for the five sizes and two thumb types. Figure 6. View largeDownload slide Mean error rate for NT and DT use in five target sizes (left) and three target distances (right). Error bars represent 0.95 confidence interval. Figure 6. View largeDownload slide Mean error rate for NT and DT use in five target sizes (left) and three target distances (right). Error bars represent 0.95 confidence interval. Table 2. Mean error rates and SD for NT and DT use in five target sizes and three target distances. Target size (mm) Target distance (mm) 3.8 5.8 7.7 9.6 11.5 21 39 58 NT Mean (%) 12.6 5.3 2.6 1.0 0.5 2.5 4.5 6.2 SD (%) 4.5 3.2 1.4 0.6 0.4 1.2 2.2 3.1 DT Mean (%) 11.4 4.5 2.1 1.0 0.5 2.1 4.1 5.5 SD (%) 4.8 3.4 1.2 0.5 0.3 1.3 2.2 3.2 Target size (mm) Target distance (mm) 3.8 5.8 7.7 9.6 11.5 21 39 58 NT Mean (%) 12.6 5.3 2.6 1.0 0.5 2.5 4.5 6.2 SD (%) 4.5 3.2 1.4 0.6 0.4 1.2 2.2 3.1 DT Mean (%) 11.4 4.5 2.1 1.0 0.5 2.1 4.1 5.5 SD (%) 4.8 3.4 1.2 0.5 0.3 1.3 2.2 3.2 View Large Table 2. Mean error rates and SD for NT and DT use in five target sizes and three target distances. Target size (mm) Target distance (mm) 3.8 5.8 7.7 9.6 11.5 21 39 58 NT Mean (%) 12.6 5.3 2.6 1.0 0.5 2.5 4.5 6.2 SD (%) 4.5 3.2 1.4 0.6 0.4 1.2 2.2 3.1 DT Mean (%) 11.4 4.5 2.1 1.0 0.5 2.1 4.1 5.5 SD (%) 4.8 3.4 1.2 0.5 0.3 1.3 2.2 3.2 Target size (mm) Target distance (mm) 3.8 5.8 7.7 9.6 11.5 21 39 58 NT Mean (%) 12.6 5.3 2.6 1.0 0.5 2.5 4.5 6.2 SD (%) 4.5 3.2 1.4 0.6 0.4 1.2 2.2 3.1 DT Mean (%) 11.4 4.5 2.1 1.0 0.5 2.1 4.1 5.5 SD (%) 4.8 3.4 1.2 0.5 0.3 1.3 2.2 3.2 View Large We further looked into error rate data for each thumb use condition by considering target size factors. Not surprisingly, target sizes had a significant main effect on error rates for NT use ( F4,44=35.42,P<0.001) and for DT use ( F4,44=32.12,P<0.001), with larger sizes having lower error rates. For the NT condition, no pairwise differences were found between 5.8 and 7.7 mm, and between 9.6 and 11.5 mm. And for the DT condition, no pairwise differences were found among 7.7, 9.6 and 11.5 mm. 3.7. Discussion In this section, we discuss the experimental results around the questions raised before the experiment. Q1 Would the differences and similarities between DT and NT depend on target sizes and distances? A1 NT and DT differences depended on target sizes and target distances. While DT had generally better performance than NT, they had very similar performance for large target sizes (9.6 and 11.5 mm) and short target distances (21 mm). DT has greater dexterity than NT, hence leading to higher average speed; However, such differences did not consistently exist when target sizes became larger or target distances became smaller. Q2 What is the required target size for fast and accurate pointing with NT and DT on tablets respectively? A2 The required target size for NT and DT pointing is different. For NT input, 9.6 and 11.5 mm sizes resulted in significantly shorter time and lower error rates than other sizes, and no significant differences were found between 9.6 and 11.5 mm sizes. Therefore, target size of 9.6 mm would be large enough to support fast and accurate pointing by NT. For DT use, 7.7, 9.6 and 11.5 mm sizes achieved significantly shorter pointing time and lower error rates than other sizes; while the differences between the three sizes were not statistically significant. As a result, target size of 7.7 mm would be sufficiently large for fast and accurate DT pointing. Overall, NT needs larger target size than DT for fast and accurate tapping performance. As done in Parhi et al. (2006), we further checked the required target size by analyzing on-screen hit distribution for target size of 9.6 mm (NT) and 7.7 mm (DT) in all 15 screen locations. We calculated the maximum diameter of any of the 2-SD bounding boxes to gain the minimum sized box that would be expected to enclose 95% of hits at the 15 screen locations. We found the required target size for NT is reduced to 9.4 mm but size for DT remains 7.7 mm. The smaller target size required by DT tapping may be because DT is more skilled in tapping targets. Also note in comparison to the study (Parhi et al., 2006) which revealed the optimal target size for one-handed dominant thumb pointing was 9.2 mm, the target size for two-handed dominant thumb pointing was smaller. Holding a mobile device with two hands is more stable than with one, which may account for why the required size shrank in our study. 4. EXPERIMENT TWO: INVESTIGATING DOMINANT VS. NON-DOMINANT THUMB GESTURING In this experiment, we examined the differences and similarities between DT and NT gesturing. We focused on stroke gestures, as this is a widely studied gesture type (e.g. Bragdon et al., 2011; Cao and Zhai, 2007; Tu et al., 2015). To find their intrinsic differences, we adopted the methodology in the study of pen vs. finger gestures (Tu et al., 2015) and designed the experiment accordingly. 4.1. Questions for investigation We would like to address the following three questions of DT and NT gesturing. Q3 Would DT and NT gestures differ in time and accuracy? We anticipated DT should outperform NT in time and accuracy for gesturing tasks as the dominant thumb has greater dexterity. Q4 Would the differences of time and accuracy between DT and NT gestures depend on gesture complexity and target gesture sizes? As having higher degrees of flexibility, DT should be good at drawing more complex gestures and smaller gestures in comparison with NT. Q5 How would DT gestures differ from NT gestures regarding local and global shape features? DT gestures should be more precise in local details than NT gestures due to higher degrees of dexterity involved in DT use. NT should be just as effective as DT at producing global shape features since these features are scale independent. 4.2. Gesture categories In order to identify differences between DT and NT gesturing, we used the same set of gesture prototypes in Tu et al. (2015). Our goal was to have a small gesture set that covers a wide range of gestures across different categories. While these gestures were initially selected to compare pen and finger gestures, they are representative samples with varied complexities and should be adequate for the purpose of this study as well. As illustrated in Table 3, these gestures were classified into simple, medium and complex categories according to their visual appearance (i.e. the number of corners, curves and line segments), gesture length, estimated production time (calculated by the CLC model, Cao and Zhai, 2007). Table 3. Prototype gestures used in the experiment. View Large Table 3. Prototype gestures used in the experiment. View Large These gestures also vary in geometrical characteristics in a variety of ways. Gestures G1, G2, G5, G6, G9 and G10 are composed of corners and straight lines (polylines), and Gestures G3, G4, G7, G8, G11 and G12 are mainly composed of curves. Gestures G1, G3, G5, G7, G9 and G11 are closed gestures because their starting and ending points are the same. The rest of the gestures in the set are open gestures. Gestures G1, G3, G4, G5, G7, G9 and G11 are symmetrical about the Y axis, and the others are asymmetrical. 4.3. Presentation size of prototype gestures (target gesture size) DT and NT vary in the dexterity required to perform gesturing tasks. We suspected that DT may be easier to draw small gestures. We hence repeated the same set of gestures in two different presentation sizes and ask the participants to reproduce them accordingly. Following the previous work (Tu et al., 2015), we defined the target gesture size (i.e. the presented) as the area in cm2 of the gesture’s bounding box, and selected two target gesture sizes ( 1.5×1.5cm and 3.0×3.0cm). We did not use 4.5×4.5cm size because our pilot study showed it was difficult and uncomfortable for participants to draw gestures with DT and NT in such a size. 4.4. Experiment apparatus The same equipment was used as in Experiment 1. 4.5. Participants Fourteen unimpaired right-handed participants (7 male) with mean ( ±SD) age of 24.0±3.2yrs and thumb length of 6.2±0.6cm took part in the experiment. All had experience in using touchscreen devices with 3.6 years on average. All had used tablet devices for 1 year on average. None of them participated in Experiment 1. 4.6. Experiment design Our experiment used a within subjects repeated measures design and included three independent variables: thumb types (DT and NT), gesture complexity (simple, medium and complex) and target gesture sizes ( 1.5×1.5cm and 3.0×3.0cm). The dependent variables are introduced in Section 4.8 for a better organization of the content. 4.7. Task and procedure This experiment adopted a recall-based gesture production task rather than visual contour tracing task, as the task mimicked the gesture behavior on touchscreen devices in practice. Figure 7a and d shows the interface at the start of a trial. After a target gesture was shown in the middle frame for 1.5 seconds,1 the gesture disappeared (Fig. 7b and e) and the participant was asked to draw a corresponding gesture from memory as quickly and accurately as possible, using the right thumb (Fig. 7c) or the left thumb (Fig. 7f). Figure 7. View largeDownload slide Experiment interface for right thumb input (a–c) and for left thumb input (d–f). (a) and (d): the start of a trial; (b) and (e): prototype gesture disappears, indicating participants to draw a gesture; (c) and (f): the participant has drawn a gesture with the corresponding thumb. Figure 7. View largeDownload slide Experiment interface for right thumb input (a–c) and for left thumb input (d–f). (a) and (d): the start of a trial; (b) and (e): prototype gesture disappears, indicating participants to draw a gesture; (c) and (f): the participant has drawn a gesture with the corresponding thumb. The experiment consisted of a practice phase and a test phase. In the practice phase, participants were first instructed of how to perform the experiment. Then they were asked to sit in a chair and hold the device with two hands in the landscape view. Participants practiced 24 trials (12 prototype gestures × 2 target gesture sizes) by the left and right thumbs respectively. The order of trials was randomized. In the test phase, each participant completed three blocks for each thumb use. As practice, each block had 24 randomly ordered trials. The order of using left or right thumb was counterbalanced across participants. The participants on average took 25 minutes to finish the experiment. In summary, the experiment consisted of (excluding practice trials) 12 participants × 24 trials × 2 thumb types × 3 blocks=1728 trials. 4.8. Results and analysis Repeated measures ANOVA and post hoc comparisons with the Bonferroni adjustment were used in the data analysis. We first checked the learning effect on stroke articulation time over the three blocks of trials to see if the data collected had reached a level of stability. Articulation time was defined as the duration from the moment the thumb touched the screen to the moment the thumb was lifted from the screen. No significant differences were found among the three blocks for both left and thumb input ( p>0.05). After the practice phase, participants had reached a steady performance. Therefore, we used the three blocks’ data in the rest of data analysis. Based on Tu et al. (2015), we used six features to examine the differences between DT and NT gestures. These features fell into two dimensions: algebraical property feature and geometric shape feature (Table 4). The former one represents basic features of a gesture, including articulation time and size ratio. The latter one consists of local shape features and global shape geometry features. It is focused on shape characteristics of a gesture. DT and NT use may lead to different performances due to dexterity differences in terms of these features. Table 4. Feature categories. Feature categories Measures Features Algebraical property Basic measure 1. Articulation time 2. Gesture size ratio Geometric shape Local shape measure 3. Aperture between the start point and the end point of closed gestures 4. Indicative angle difference between drawn gesture and target gesture Global shape measure 5. Axial symmetry 6. Proportional shape distance Feature categories Measures Features Algebraical property Basic measure 1. Articulation time 2. Gesture size ratio Geometric shape Local shape measure 3. Aperture between the start point and the end point of closed gestures 4. Indicative angle difference between drawn gesture and target gesture Global shape measure 5. Axial symmetry 6. Proportional shape distance View Large Table 4. Feature categories. Feature categories Measures Features Algebraical property Basic measure 1. Articulation time 2. Gesture size ratio Geometric shape Local shape measure 3. Aperture between the start point and the end point of closed gestures 4. Indicative angle difference between drawn gesture and target gesture Global shape measure 5. Axial symmetry 6. Proportional shape distance Feature categories Measures Features Algebraical property Basic measure 1. Articulation time 2. Gesture size ratio Geometric shape Local shape measure 3. Aperture between the start point and the end point of closed gestures 4. Indicative angle difference between drawn gesture and target gesture Global shape measure 5. Axial symmetry 6. Proportional shape distance View Large 4.8.1. Basic measures Articulation time. The definition of articulation time can be found in the first paragraph of this subsection. It is a basic metric to measure gesture performance. Thumb types had a significant main effect on articulation time ( F1,13=38.22,P<0.001). The mean time for NT and DT was 2342 and 2091 ms, respectively. Gesture sizes and complexities also had significant main effects on articulation time ( F1,13=83.54,P<0.001 and F2,26=458.18,P<0.001, respectively). While there was no significant interaction effect between thumb types and gesture sizes ( F1,13=0.69,P=0.42), a significant interaction effect was found between thumb types and gesture complexities ( F2,26=19.52,P<0.001) (Fig. 8). The difference between NT and DT increased from simple to complex gestures (Table 5). Figure 8. View largeDownload slide Articulation time for each implement in three complexities (left) and two target gesture sizes (right). Error bars represent 0.95 confidence interval. Figure 8. View largeDownload slide Articulation time for each implement in three complexities (left) and two target gesture sizes (right). Error bars represent 0.95 confidence interval. Table 5. Mean values of the features for DT and NT in three complexities and two target gesture sizes. Features Mean value (NT vs. DT) Complexity (NT vs. DT) Target gesture size (NT vs. DT) Simple Medium Complex 1.5×1.5cm 3.0×3.0cm Articulation time (sec.) 2.34 vs. 2.09 1.44 vs. 1.29 2.22 vs. 2.00 3.37 vs. 2.99 2.21 vs. 1.93 2.48 vs. 2.25 Gesture size ratio 1.34 vs. 1.40 1.15 vs. 1.20 1.31 vs. 1.37 1.57 vs. 1.62 1.64 vs. 1.67 1.05 vs. 1.12 Aperture (cm) 0.22 vs. 0.21 N/A N/A N/A 0.21 vs. 0.19 0.23 vs. 0.22 Angular difference (degree) −1.26 vs. 3.60 N/A N/A N/A −1.61 vs. 3.58 −0.91 vs. 3.84 Axial symmetry 0.59 vs. 0.55 0.37 vs. 0.35 0.50 vs. 0.49 0.90 vs. 0.81 0.62 vs. 0.60 0.56 vs. 0.50 Shape distance (cm) 0.44 vs. 0.43 0.33 vs. 0.32 0.41 vs. 0.39 0.60 vs. 0.57 0.48 vs. 0.47 0.40 vs. 0.39 Features Mean value (NT vs. DT) Complexity (NT vs. DT) Target gesture size (NT vs. DT) Simple Medium Complex 1.5×1.5cm 3.0×3.0cm Articulation time (sec.) 2.34 vs. 2.09 1.44 vs. 1.29 2.22 vs. 2.00 3.37 vs. 2.99 2.21 vs. 1.93 2.48 vs. 2.25 Gesture size ratio 1.34 vs. 1.40 1.15 vs. 1.20 1.31 vs. 1.37 1.57 vs. 1.62 1.64 vs. 1.67 1.05 vs. 1.12 Aperture (cm) 0.22 vs. 0.21 N/A N/A N/A 0.21 vs. 0.19 0.23 vs. 0.22 Angular difference (degree) −1.26 vs. 3.60 N/A N/A N/A −1.61 vs. 3.58 −0.91 vs. 3.84 Axial symmetry 0.59 vs. 0.55 0.37 vs. 0.35 0.50 vs. 0.49 0.90 vs. 0.81 0.62 vs. 0.60 0.56 vs. 0.50 Shape distance (cm) 0.44 vs. 0.43 0.33 vs. 0.32 0.41 vs. 0.39 0.60 vs. 0.57 0.48 vs. 0.47 0.40 vs. 0.39 N/A means gesture complexity cannot serve as an independent variable for the interaction effect analysis. View Large Table 5. Mean values of the features for DT and NT in three complexities and two target gesture sizes. Features Mean value (NT vs. DT) Complexity (NT vs. DT) Target gesture size (NT vs. DT) Simple Medium Complex 1.5×1.5cm 3.0×3.0cm Articulation time (sec.) 2.34 vs. 2.09 1.44 vs. 1.29 2.22 vs. 2.00 3.37 vs. 2.99 2.21 vs. 1.93 2.48 vs. 2.25 Gesture size ratio 1.34 vs. 1.40 1.15 vs. 1.20 1.31 vs. 1.37 1.57 vs. 1.62 1.64 vs. 1.67 1.05 vs. 1.12 Aperture (cm) 0.22 vs. 0.21 N/A N/A N/A 0.21 vs. 0.19 0.23 vs. 0.22 Angular difference (degree) −1.26 vs. 3.60 N/A N/A N/A −1.61 vs. 3.58 −0.91 vs. 3.84 Axial symmetry 0.59 vs. 0.55 0.37 vs. 0.35 0.50 vs. 0.49 0.90 vs. 0.81 0.62 vs. 0.60 0.56 vs. 0.50 Shape distance (cm) 0.44 vs. 0.43 0.33 vs. 0.32 0.41 vs. 0.39 0.60 vs. 0.57 0.48 vs. 0.47 0.40 vs. 0.39 Features Mean value (NT vs. DT) Complexity (NT vs. DT) Target gesture size (NT vs. DT) Simple Medium Complex 1.5×1.5cm 3.0×3.0cm Articulation time (sec.) 2.34 vs. 2.09 1.44 vs. 1.29 2.22 vs. 2.00 3.37 vs. 2.99 2.21 vs. 1.93 2.48 vs. 2.25 Gesture size ratio 1.34 vs. 1.40 1.15 vs. 1.20 1.31 vs. 1.37 1.57 vs. 1.62 1.64 vs. 1.67 1.05 vs. 1.12 Aperture (cm) 0.22 vs. 0.21 N/A N/A N/A 0.21 vs. 0.19 0.23 vs. 0.22 Angular difference (degree) −1.26 vs. 3.60 N/A N/A N/A −1.61 vs. 3.58 −0.91 vs. 3.84 Axial symmetry 0.59 vs. 0.55 0.37 vs. 0.35 0.50 vs. 0.49 0.90 vs. 0.81 0.62 vs. 0.60 0.56 vs. 0.50 Shape distance (cm) 0.44 vs. 0.43 0.33 vs. 0.32 0.41 vs. 0.39 0.60 vs. 0.57 0.48 vs. 0.47 0.40 vs. 0.39 N/A means gesture complexity cannot serve as an independent variable for the interaction effect analysis. View Large Gesture size ratio. Participants may draw gestures in larger or smaller scales with reference to the target gesture displayed. The size ratio between the drawn gesture and the target gesture can therefore reflect users’ ability of drawing gestures at a specified scale. There was a significant main effect on size ratio for thumb types ( F1,13=4.47,P<0.05). The mean size ratio for NT (1.34) was slightly smaller than that for DT (1.40). Gesture sizes and complexities had significant main effects on size ratio ( F1,13=99.06,P<0.001 and F2,26=87.87,P<0.001 for gesture sizes and complexities respectively). No significant interaction effects were found for thumb types × gesture sizes ( F1,13=2.49,P=0.14) and for thumb types × gesture complexities ( F2,26=0.08,P=0.93). The mean values can be found in Table 5. 4.8.2. Local shape measures Aperture between the start point and the end point of closed gestures. The prototype gestures G1, G3, G5, G7, G9 and G11 start and end in the same position (Table 3). To represent the ability to draw a closed gesture corresponding to these prototype gestures, we measured the distance (aperture, d) between the start point ( p) and the end point ( q).2 d(p,q)=(px−qx)2+(py−qy)2. (1) No significant main effect was found on aperture for thumb types ( F1,13=1.82,P=0.20). However, there was a significant main effect for gesture sizes ( F1,13=15.36,P<0.01). Small gesture sizes tended to have smaller apertures than large gesture sizes. There was no significant interaction effect between thumb types and gesture sizes ( F1,13=0.17,P=0.69). Indicative angle difference between drawn gestures and target gestures. Following Wobbrock et al. (2007), the indicative angle was calculated as anticlockwise rotation from the horizontal vector whose starting point is the centroid of the gesture, to the vector formed by the centroid of the gesture and the gesture’s first point (left image of Fig. 9: θ). It indicates the orientation of a gesture. We calculated the indicative angle difference between the drawn gesture and the corresponding target gesture. Figure 9. View largeDownload slide Left: illustration of the indicative angle θ of G4. Right: illustration of axial symmetry for a drawn gesture corresponding to G4. Figure 9. View largeDownload slide Left: illustration of the indicative angle θ of G4. Right: illustration of axial symmetry for a drawn gesture corresponding to G4. There was a significant main effect for thumb types ( F1,13=17.50,P<0.01). The mean angle difference for the non-dominant thumb was −1.26° while the mean value for the dominant thumb was 3.60°. No significant main effect was found for gesture sizes ( F1,13=1.75,P=0.21). No significant interaction effects were found for thumb types × gesture sizes ( F1,13=0.06,P=0.81). 4.8.3. Global shape measures We used the same method in Tu et al. (2015) to investigate global shape aspects of a drawn gesture. We disregarded gesture sizes by normalizing ( scaling) both the drawn gesture’s size and the corresponding target gesture’s size to 4.5×4.5cm. If the drawn gesture has the exact relative dimensions as the target gesture, the normalized shape measures would have zero-distance difference. Axial symmetry. The prototype gestures G1, G3, G4, G5, G7, G9 and G11 (Table 3) have axial symmetry. We would like to examine how users draw these gestures with DT and NT in terms of axial symmetry. The drawn gesture was first scaled to 4.5×4.5cm size and then re-sampled it to N ( N=500) equidistant points. The drawn gesture was divided into left and right parts by X=Xa, an axis which crosses the geometric center of the drawn gesture and is perpendicular with the X axis (right image of Fig. 9). To check the axial symmetry of the drawn gesture, we calculated the distance difference between its left and right parts. For straight lines Y=Yi ( Ymin≤Yi≤Ymax, Ymin and Ymax are the minimum y value and the maximum y value of the drawn gesture, respectively, Yi increases 1 pixel each time), there are two intersecting points between the drawn gesture and Y=Yi: ( Xa−XL, Yi) in the left of X=Xa and ( Xa+XR, Yi) in the right of X=Xa (right image of Fig. 9), where XL is the distance between X=Xa and ( Xa−XL, Yi), XR is the distance between X=Xa and ( Xa+XR, Yi). The mean distance difference is AS=1Ymax−Ymin∑i=YminYmaxDAi (2) where DAi is the absolute value of ( XR−XL). The greater the AS is, the less symmetrical the drawn gesture is. There was no significant main effect for thumb types ( F1,13=2.48,p=0.14). However, a significant main effect was found for gesture sizes ( F2,26=63.15,P<0.01). While there were no significant interaction effects for thumb types × gesture complexities ( F2,26=1.41,P=0.26), a significant interaction effect was found for thumb types × gesture sizes ( F1,13=5.76,P<0.05). The mean values are illustrated in Table 5. Proportional shape distance. Proportional shape distance (PSD) is a widely used measure in gesture recognition algorithms and techniques (e.g. the ShapeWriter gesture keyboard) (Wobbrock et al., 2007; Zhai and Kristensson, 2003). We examined this measure in a simpler way in this study. After scaling, the drawn gesture was translated to make its centroid coincides with the centroid of the target gesture. Then the drawn gesture U and the target gesture V were re-sampled into N ( N=100) evenly spaced points denoted by U(i) and V(i) ( 1≤i≤N) for U and V, respectively. PSD was defined as PSD=1N∑i=1Nd(U(i),V(i)) (3) where d(U(i),V(i)) means the distance between the point U(i) and the point V(i) (Formula 1). We suspected that DT would outperform NT due to the higher flexibility of DT, so DT gestures are supposed to have a smaller PSD. However, no significant main effect was found for thumb types ( F1,13=1.94,P=0.19). Given this feature was always to indicate gesture articulation accuracy, DT and NT therefore resulted in similar accuracy when drawing gestures. There was a significant main effect for gesture sizes ( F1,13=144.61,P<0.001) and gesture complexities ( F2,26=175.20,P<0.001). Target gestures with larger size and higher complexity tended to have larger PSD value. While there were no significant interaction effects for thumb types × gesture sizes ( F1,13=0.01,P=0.92), a significant interaction effect was found for thumb types × gesture complexities ( F2,26=17.27,P<0.01) (Fig. 10). Table 5 shows the mean values. Figure 10. View largeDownload slide PSD in normalized scale for each implement in three complexities (left) and two target gesture sizes (right). Error bars represent 0.95 confidence interval. Figure 10. View largeDownload slide PSD in normalized scale for each implement in three complexities (left) and two target gesture sizes (right). Error bars represent 0.95 confidence interval. 4.9. Discussion In this section, we discuss the experimental results against the expectations and questions listed at the beginning of this experiment. Q3 Would DT and NT drawn gestures differ in time and accuracy? A3 Results reveal that DT achieved shorter articulation time than NT in drawing simple, medium and complex gestures. However, differences were not found in gesture production accuracy according to the analysis of PSD. Therefore, the different degrees of dexterity between DT and NT affected gesture input time, but not input accuracy. Q4 Would the differences of time and accuracy between DT and NT gestures depend on gesture complexity and target gesture sizes? A4 The differences between DT and NT gestures depended on gesture complexity. DT input resulted in shorter articulation time and smaller PSD than NT input, but the difference decreased from complex gestures to simple gestures. This indicates gesture set design for NT input should not contain gestures which are overly complex. However, the differences between DT and NT gestures did not depend on gesture sizes, as there were no significant interaction effects between thumb types and gesture sizes for articulation time and PSD. Q5 How would DT gestures differ from NT gestures regarding local and global shape features? A5 For local features like aperture and global features such as axial symmetry and PSD, DT and NT drawn gestures were overall quite similar. While DT has higher degrees of dexterity involved in drawing gestures, it seems that this advantage did not have significant effects on articulating most shape features of gestures. However, DT and NT drawn gestures differed in indicative angle difference. When drawing a directional gesture like a line from left to right, participants needed to stretch the left thumb or bend the right thumb to complete the task. This movement contrast may explain why there was a significant difference between DT and NT in terms of indicative angle difference. 5. GENERAL DISCUSSION This study has uncovered many notions regarding NT and DT input for bimanual tablet gripping interaction. We discuss the results with the implications below. 5.1. Promote understanding of differences between NT and DT use Studies have revealed many differences between the dominant and non-dominant hands in pointing, selecting and dragging tasks (Kabbash et al., 1993). The present study is focused on the comparison of pointing and gesturing between the left and right thumbs for bimanual tablet gripping interaction, and gains many differences of their motor control abilities unveiled before. While DT had better performance than NT in general, their differences varied by conditions of some independent variables in our experiments. For pointing tasks, DT and NT had significant differences in pointing time if target sizes were smaller than 7.7 mm or if target distances were larger than 39 mm. However, no significant differences existed if target sizes were larger than 9.6 mm or if target distances were smaller than 21 mm. Such results are consistent with previous Fitts’ reciprocal tapping study (Flowers, 1975) which revealed that the dominant and non-dominant hands did not differ in low IDs. One possible reason is that thumb movement is restricted by the frame of the tablet, which may eliminate the motor control differences between DT and NT. For gesturing tasks, DT input resulted in shorter articulation time and smaller shape distance than NT input on average, but the difference decreased from complex gestures to simple gestures. This indicates it is better to avoid using overly complex gestures for NT input. There are many other differences between DT and NT input. For pointing tasks, we examined the required target size for fast and accurate pointing with DT and NT respectively. The required target size was 9.4 mm in diameter for NT use, which is larger than DT use (7.7 mm in diameter). If targets are larger than such sizes, it is reasonable to expect thumb pointing performance would not degrade. For gesturing tasks, DT and NT differed in features like articulation time, size ratio, and indicative angle difference. If gesture interface design for bimanual thumb input depends on these features, their effects on gesture performance for DT and NT may vary. Therefore, gesture interface design should either avoid using these features for NT and DT input or carefully evaluate the effects of using these features. For example, according to the analysis of size ratio, drawn gestures by NT were smaller than DT generally. Hence, if designers want to use gesture scale (i.e. small, medium, and large) to enter discrete parameters for a command for NT or DT (e.g. a small ‘S’ for ‘Save’ command and a large ‘S’ for ‘Save As’ command as shown in Vatavu et al., 2013), they need to consider the ability of articulating stroke gestures at various scales using DT and NT, respectively. Overall, identifying these differences can aid user interface design for bimanual tablet gripping interaction. Our analysis gains insights into many different characteristics between DT and NT interactions. Some results were easy to anticipate according to previous studies and our daily experiences on using dominant and non-dominant hands (e.g. DT had overall better performance than NT), while others were first revealed through our quantitative evaluation (e.g. DT and NT pointing were not significantly different for large targets (9.6 and 11.5 mm) and near distance (21 mm)). Many factors may contribute to the differences. From the view of human anatomy, DT is inverted in its physical structure from that of NT, hence may influence gesture articulation. For example, when drawing a directional gesture like a line from left to right, participants need to stretch the left thumb or flex the right thumb to complete the task. This movement contrast may cause performance differences. In addition, the dexterity of DT is unmatchable by NT. As a result, a pointing or gesturing task that lends itself to DT may be awkward to execute on NT. 5.2. Increase awareness of similarities between NT and DT input Our study reveals many similar aspects between NT and DT input as below. Participants were able to perform and achieve a similar level of proficiency when performing pointing tasks with either thumb if target sizes are larger than 9.4 mm. From this perspective, NT is more than a poor approximation of DT; they are complementary and can be used interchangeably. In most implementations of Guiard’s theory (Guiard, 1987), the preferred hand performs precise actions and the non-preferred hand is restricted to coarse, simpler actions or used as a frame of reference. According to the results in our study, designers may thus need to reconsider their adherence to Guiard’s theory. For bimanual tablet gripping interaction, users are able to perform pointing-based interactions efficiently for large targets ( ≥9.6mm) using either thumb. This result not only enriches Guard’s theory in the context of bimanual tablet gripping interaction, but also has design implication—interface elements such as icons, menu items in such size can be arranged on either hand side when designing interfaces for bimanual tablet gripping interaction. It has two benefits to delegate pointing-based interaction to both thumbs. First, the ability to perform pointing tasks with either thumb is one way to mitigate hand fatigue on tablet devices. Second, it entails more interaction possibilities for bimanual tablet gripping interaction, as both thumbs can be used interchangeably to execute pointing tasks with comparable efficacy if targets reach 9.6 mm size. Gestures drawn by the two thumbs were not significantly different in terms of local and global shape features such as aperture, axial symmetry, and PSD. If gesture interface design is based on these features, NT gestures should be as effective as DT gestures. Therefore, these features can be equally applied to DT and NT friendly gesture interface design. For example, given that PSD-based recognition has already been used in both research and practical large scale gesture systems (specifically the ShapeWriter gesture keyboard (Zhai and Kristensson, 2003), although in more complex ways than in this paper), it is reasonable to expect that gesture recognition algorithm can be designed for either thumb use within the feature space outlined in this paper. 5.3. Examples of design applications The discussion above provides many implications to user interface design for bimanual tablet gripping interaction. Generally speaking, NT and DT input has many different and similar characteristics, and designers and researchers should be mindful of these characteristics when designing and implementing future bimanual tablet interfaces. Here we list two specific design examples as follows. Interface element sizes for NT and DT pointing—to achieve fast and accurate pointing performance, the required size of interface elements like buttons should be 9.4 mm in diameter for NT use and 7.7 mm in diameter for DT use. If interface elements are designed to be larger than 9.6 mm, they can be arranged on either side of the display in the thumb’s active area (i.e. the area that the thumb can comfortably reach without ‘breaking’ the grip Odell and Chandrasekaran, 2012), as pointing could be completed efficiently with both thumbs. This guideline can be directly applied to tablet application design like games where directional pad and action buttons commonly affix to the two sides of the display. Recognition algorithms for NT and DT gesturing—PSD-based recognition can be equally applied to DT or NT drawn gestures. If gesture interfaces rely on such recognition, gestures drawn by either thumb should be treated to the same degree for recognition. In this regard, there is no need to identify which thumb is used for gesturing and to modify recognition algorithm parameters to accommodate differences between NT and DT gestures. This is quite meaningful for most current tablet devices which cannot detect handedness. 6. LIMITATIONS AND FUTURE WORK While our study gains valuable results for bimanual tablet interface design, there are some directions that can be pursued to extend the current work. 6.1. Grasping styles We did not include gripping styles as an experimental condition given the limited manageable scope of one controlled study. As many previous studies (e.g. Bi et al., 2012; Odell and Chandrasekaran, 2012; Wolf and Henze, 2014), we asked the participants to grasp the sides of the tablet comfortably but did not specify which gripping style (e.g. clamping or cradling) during the experiment process. The aim was to let the participants naturally hold the device to perform the task. We observed they tended to adopt clamping as shown in Fig. 1. This is identical to the best-performing grip for thumb tapping identified by Oulasvirta et al. (2013). With such gripping style, the tablet’s edge is on the thenar crease, hence locking the more distal joints of the hand and providing a stable hold. Furthermore, such gripping style relies on the three joints of the thumb (i.e. IP, MPC and CMC joints) for tapping, so supports faster thumb movement than cradling (Oulasvirta et al., 2013). While our findings are based on a ‘natural’ griping style by participants, it is of interests to examine the performance of NT versus DT on tablet with other gripping styles such as cradling. With cradling, the performance of pointing and gesturing for each thumb may vary from the results presented in this paper, as such style allows control by the more distal palmar muscles. However, this should not significantly affect the differences between NT and DT, because the control behavior of both thumbs can be symmetrically adjusted. Future work includes an empirical investigation to demonstrate this point. Besides the symmetric grip examined in this study, there are asymmetric grips for bimanual tablet interaction. For example, users can clamp the device with their non-dominant hand, and interact with it by their dominant hand or the thumb of their non-dominant hand (Wagner et al., 2012). It needs further investigation to check if the conclusions in our study still hold true for such griping styles. We leave this as future work. 6.2. Device forms Device forms are another factor worthy of note. We evaluated NT vs. DT on a Google Nexus 10 multitouch tablet with 10.1 inches in size. Such tablet size is representative as many popular tablet devices had the similar size, e.g. Samsung Galaxy Tab (10.1 inches), iPad (9.7 inches), Lenovo Yoga 10 (10.1 inches) (see Tablet PC Sizes, 2018 for more). For bimanual gripping interaction on tablets in other forms, the conclusions should remain consistent since device forms should not significantly impact on motor control ability of both thumbs. 6.3. Interaction tasks In this study, we compared the performance between NT and DT in executing pointing and gesturing tasks, given the two tasks are more common than other tasks such as crossing and writing for bimanual tablet interaction. Future work will further evaluate the differences and similarities between NT and DT for crossing and writing task, so as to achieve a more comprehensive understanding towards bimanual thumb interaction. 6.4. Handedness To avoid bias caused by handedness, we deliberately chose right-handed participants in our experiments. Since the structures of left and right hands are symmetrical, we expect that the conclusions in this study would be applicable to left-handed users as well. The comparison of experiment design with left-handed and right-handed participants is beyond the scope of our study and will be examined in future work. 7. CONCLUSION The proliferation of tablet devices creates challenges to user interface design for them. As a common interaction style, bimanual thumb interaction on tablets has several benefits but the differences between the two thumb use are still unclear. Hence, we conducted a first study to evaluate the differences and similarities between DT and NT input for pointing and gesturing interaction. We found that NT input differed from DT input in many aspects. For both tasks, NT input was significantly slower than DT input. The required target size for fast and accurate pointing with DT and NT was different (7.7 mm and 9.4 mm in diameter for DT and NT). DT and NT were different in gesture features like articulation time, size ratio and indicative angle difference. On the other hand, DT and NT were quite similar in other aspects such as pointing and gesturing accuracy. Pointing performance of DT and NT were comparable for large targets and near targets. Gestures drawn by the two thumbs were quite alike in gesture features such as aperture, axial symmetry, and PSD. The results provide empirical evidence of the significance to evaluate two-handed thumb pointing and gesturing performance on tablet devices and would be of value to future bimanual thumb interaction design and research. FUNDING This work is supported by the National Key R&D Program of China (No. 2016YFB1001402); Key Research Program of Frontier Sciences, CAS (No. QYZDY-SSW-JSC041); Natural Science Foundation of China (No. 61602236); Science Foundation of Jiangsu Province (No. BK20160801); China Postdoctoral Science Foundation (No. 2016M591843); and Postdoctoral Science Foundation of Jiangsu Province (No. 1501053B). REFERENCES Annett , J. , Annett , M. , Hudson , P.T.W. and Turner , A. ( 1979 ) The control of movement in the preferred and non-preferred hands . Q. J. Exp. Psychol. , 31 , 641 – 652 . Google Scholar CrossRef Search ADS PubMed Azenkot , S. and Zhai , S. ( 2012 ) Touch Behavior with Different Postures on Soft Smartphone Keyboards. In Proc. 14th Int. Conf. Human-computer Interaction with Mobile Devices and Services, pp. 251–260. ACM. Balakrishnan , R. and Hinckley , K. ( 2000 ) Symmetric Bimanual Interaction. In Proc. SIGCHI Conf. Human Factors in Computing Systems, pp. 33–40. ACM. Bi , X. , Chelba , C. , Ouyang , T. , Partridge , K. and Zhai , S. ( 2012 ) Bimanual Gesture Keyboard. In Proc. 25th Annual ACM Symposium on User Interface Software and Technology, pp. 137–146. ACM. Bragdon , A. , Nelson , E. , Li , Y. and Hinckley , K. ( 2011 ) Experimental Analysis of Touch-screen Gesture Designs in Mobile Environments. In Proc. SIGCHI Conf. Human Factors in Computing Systems, pp. 403–412. ACM. Buxton , W. , Billinghurst , M. , Guiard , Y. , Sellen , A. and Zhai , S. ( 2017 ) Human input to computer systems: theories, techniques and technology. Book manuscipt. Retrieved in January, 2018. https://www.billbuxton.com/inputManuscript.html. Buxton , W. and Myers , B. ( 1986 ) A Study in Two-handed Input. In ACM SIGCHI Bulletin (Vol. 17, No. 4, pp. 321–326). ACM. Cao , X. and Zhai , S. ( 2007 , April). Modeling Human Performance of Pen Stroke Gestures. In Proc. SIGCHI Conf. Human Factors in Computing Systems, pp. 1495–1504. ACM. Clarkson , E. , Lyons , K. , Clawson , J. and Starner , T. ( 2007 , April). Revisiting and Validating a Model of Two-thumb Text Entry. In Proc. SIGCHI Conf. Human Factors in Computing Systems, pp. 163–166. ACM. Colle , H.A. and Hiszem , K.J. ( 2004 ) Standing at a kiosk: effects of key size and spacing on touch screen numeric keypad performance and user preference . Ergonomics , 47 , 1406 – 1423 . Google Scholar CrossRef Search ADS PubMed Fitts , P.M. ( 1954 ) The information capacity of the human motor system in controlling the amplitude of movement . J. Exp. Psychol. , 47 , 381 . Google Scholar CrossRef Search ADS PubMed Flowers , K. ( 1975 ) Handedness and controlled movement . Br. J. Psychol. , 66 , 39 – 52 . Google Scholar CrossRef Search ADS PubMed Guiard , Y. ( 1987 ) Asymmetric division of labor in human skilled bimanual action: the kinematic chain as a model . J. Mot. Behav. , 19 , 486 – 517 . Google Scholar CrossRef Search ADS PubMed Isokoski , P. ( 2001 ) Model for Unistroke Writing Time. In Proc. SIGCHI Conf. Human Factors in Computing Systems, pp. 357–364. ACM. Jones , L.A. and Lederman , S.J. ( 2006 ) Human Hand Function . Oxford University Press , New York . Google Scholar CrossRef Search ADS Kabbash , P. , MacKenzie , I.S. and Buxton , W. ( 1993 ) Human Performance Using Computer Input Devices in the Preferred and Non-preferred Hands. In Proc. INTERACT’93 and CHI’93 Conf. Human Factors in Computing Systems, pp. 474-481. ACM. Kin , K. , Hartmann , B. and Agrawala , M. ( 2011 ) Two-handed marking menus for multitouch devices . ACM Trans. Comput. Hum. Interact. (TOCHI) , 18 , 16 . Google Scholar CrossRef Search ADS Leganchuk , A. , Zhai , S. and Buxton , W. ( 1998 ) Manual and cognitive benefits of two-handed input: an experimental study . ACM Trans. Comput. Hum. Interact. (TOCHI) , 5 , 326 – 359 . Google Scholar CrossRef Search ADS MacKenzie , I.S. ( 1992 ) Fitts’ law as a research and design tool in human-computer interaction . Hum. Comput. Interact. , 7 , 91 – 139 . Google Scholar CrossRef Search ADS Müller , H. , Gove , J. and Webb , J. ( 2012 ) Understanding Tablet Use: A Multi-method Exploration. In Proc. 14th Int. Conf. Human-computer Interaction with Mobile Devices and Services, pp. 1–10. ACM. Odell , D. and Chandrasekaran , V. ( 2012 ) Enabling Comfortable Thumb Interaction in Tablet Computers: A Windows 8 Case Study. In Proc. Human Factors and Ergonomics Society Annual Meeting, Vol. 56, No. 1, pp. 1907–11. Sage CA: Los Angeles, CA: SAGE Publications. Oulasvirta , A. , Reichel , A. , Li , W. , Zhang , Y. , Bachynskyi , M. , Vertanen , K. and Kristensson , P.O. ( 2013 ) Improving Two-thumb Text Entry on Touchscreen Devices. In Proc. SIGCHI Conf. Human Factors in Computing Systems, pp. 2765–74. ACM. Parhi , P. , Karlson , A.K. and Bederson , B.B. ( 2006 ) Target Size Study for One-handed Thumb Use on Small Touchscreen Devices. In Proc. 8th Conf. Human-computer Interaction with Mobile Devices and Services, pp. 203–210. ACM. Perry , K.B. and Hourcade , J.P. ( 2008 ) Evaluating One Handed Thumb Tapping on Mobile Touchscreen Devices. In Proc. Graphics Interface 2008, pp. 57–64. Canadian Information Processing Society. Ren , X. and Moriya , S. ( 2000 ) Improving selection performance on pen-based systems: a study of pen-based interaction for selection tasks . ACM Trans. Comput. Hum. Interact. (TOCHI) , 7 , 384 – 416 . Google Scholar CrossRef Search ADS Tablet PC Sizes . Accessed in February 2018 . https://en.wikipedia.org/wiki/List_of_tablet_PC_dimensions_and_case_sizes Todor , J.I. and Doane , T. ( 1978 ) Handedness and hemispheric asymmetry in the control of movements . J. Mot. Behav. , 10 , 295 – 300 . Google Scholar CrossRef Search ADS PubMed Trudeau , M.B. , Asakawa , D.S. , Jindrich , D.L. and Dennerlein , J.T. ( 2016 ) Two-handed grip on a mobile phone affords greater thumb motor performance, decreased variability, and a more extended thumb posture than a one-handed grip . Appl. Ergon. , 52 , 24 – 28 . Google Scholar CrossRef Search ADS PubMed Trudeau , M.B. , Catalano , P.J. , Jindrich , D.L. and Dennerlein , J.T. ( 2013 ) Tablet keyboard configuration affects performance, discomfort and task difficulty for thumb typing in a two-handed grip . PLoS One , 8 , e67525 . Google Scholar CrossRef Search ADS PubMed Trudeau , M.B. , Udtamadilok , T. , Karlson , A.K. and Dennerlein , J.T. ( 2012 ) Thumb motor performance varies by movement orientation, direction, and device size during single-handed mobile phone use . Hum. Factors , 54 , 52 – 59 . Google Scholar CrossRef Search ADS PubMed Tu , H. , Ren , X. and Zhai , S. ( 2015 ) Differences and similarities between finger and pen stroke gestures on stationary and mobile devices . ACM Trans. Comput. Hum. Interact. (TOCHI) , 22 , 22 . Google Scholar CrossRef Search ADS Vatavu , R.D. , Casiez , G. and Grisoni , L. ( 2013 ) Small, Medium, or Large?: Estimating the User-perceived Scale of Stroke Gestures. In Proc. SIGCHI Conf. Human Factors in Computing Systems, pp. 277–280. ACM. Wagner , J. , Huot , S. and Mackay , W. ( 2012 ) BiTouch and BiPad: Designing Bimanual Interaction for Hand-held Tablets. In Proc. SIGCHI Conf. Human Factors in Computing Systems, pp. 2317–2326. ACM. Wobbrock , J.O. , Myers , B.A. and Aung , H.H. ( 2008 ) The performance of hand postures in front-and back-of-device interaction for mobile computing . Int. J. Hum. Comput. Stud. , 66 , 857 – 875 . Google Scholar CrossRef Search ADS Wobbrock , J.O. , Wilson , A.D. and Li , Y. ( 2007 ). Gestures Without Libraries, Toolkits or Training: a $1 Recognizer for User Interface Prototypes. In Proc. 20th Annual ACM Symposium on User Interface Software and Technology, pp. 159–168. ACM. Wolf , K. ( 2015 ) Grasp Interaction with Tablets . Springer , Berlin . Google Scholar CrossRef Search ADS Wolf , K. and Henze , N. ( 2014 ) Comparing Pointing Techniques for Grasping Hands on Tablets. In Proc. 16th Int. Conf. Human-computer Interaction with Mobile Devices and Services, pp. 53–62. ACM. Wolf , K. , Schleicher , R. and Rohs , M. ( 2014 ) Ergonomic Characteristics of Gestures for Front-and Back-of-tablets Interaction with Grasping Hands. In Proc. 16th Int. Conf. Human-computer Interaction with Mobile Devices and Services, pp. 453–458. ACM. Wolf , K. , Schneider , M. , Mercouris , J. and Hrabia , C.E. ( 2015 ) Biomechanics of front and back-of-tablet pointing with grasping hands . Int. J. Mobile Hum. Comput. Interact. (IJMHCI) , 7 , 43 – 64 . Google Scholar CrossRef Search ADS Zhai , S. and Kristensson , P.O. ( 2003 ) Shorthand Writing on Stylus Keyboard. In Proc. SIGCHI Conf. Human Factors in Computing Systems, pp. 97–104. ACM. Footnotes 1 Pilot studies indicated that after a practice period, this time period was long enough for participants to remember both the size and overall shape of a target gesture. 2 In the following sections, d(p,q) was used to denote the Euclidean distance between point p and point q. Author notes Editorial Board Member: Dr. Ian Oakley © The Author(s) 2018. Published by Oxford University Press on behalf of The British Computer Society. All rights reserved. For Permissions, please email: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)
TI - Differences and Similarities between Dominant and Non-dominant Thumbs for Pointing and Gesturing Tasks with Bimanual Tablet Gripping Interaction
JF - Interacting with Computers
DO - 10.1093/iwc/iwy009
DA - 2018-04-13
UR - https://www.deepdyve.com/lp/oxford-university-press/differences-and-similarities-between-dominant-and-non-dominant-thumbs-qc6okrcmIk
SP - 1
EP - 257
VL - Advance Article
IS - 3
DP - DeepDyve
ER -