Evidence for the Adoption of the Flemma as an Appropriate Word Counting Unit

Evidence for the Adoption of the Flemma as an Appropriate Word Counting Unit Abstract An important gap in the field of second language vocabulary research concerns the ability of Asian learners of English as a Foreign Language (EFL) to comprehend inflectional and derivational word family members. Japanese EFL learners (N = 279) were divided into three lexical proficiency groups, and their ability to comprehend inflectional and derivational English forms was measured with an English to Japanese translation test. A significant difference among the participants' ability to comprehend 12 base forms, associated inflected forms, and associated derived forms was found across the three proficiency groups, and even among participants who demonstrated mastery of the first 4,000 or 5,000 base forms of English. The flemma, a word's base form and associated inflectional forms, was found to be an appropriate word counting unit for most participants. Results are important because corpus research findings demonstrate that in cases where the word family provides 98 per cent coverage of texts, the flemma only provides 85 per cent coverage of the same texts. Thus, considering the detrimental impact to reading comprehension from only small decreases in the percentage of known tokens within a text, the results question the inferences made in word family-based research. INTRODUCTION An important gap in the field of second language (L2) vocabulary research concerns English as a Foreign Language (EFL) learners’ ability to comprehend inflectional and derivational forms. This is of importance as the choice of what inflectional and derivational forms are included with a word’s base form has major ramifications for the validity of corpora-derived word lists, word list-based vocabulary tests, and research based upon these (Gardner 2007). The influence of corpora and corpus-based research on educational theories, assessment, and pedagogic practices in L2 settings is significant (Nation 2001; Sinclair 2004). Vocabulary levels research, including investigations into the coverage of text high-frequency words provide, is an example of how corpus-based vocabulary investigations have shaped pedagogy. Vocabulary levels research argues that short corpora-derived lists of high-frequency words should be deliberately taught and learned because they cover such a large percentage of the tokens that English language learners encounter. Further, it is argued that learners must be able to comprehend a relatively small number of high-frequency, high-coverage words for incidental vocabulary growth; otherwise, achieving unassisted comprehension of texts and subsequently contextual word learning would be inhibited (Laufer 1997; Nation 2001). The validity of this research paradigm and corpus-based vocabulary applications is dependent on researchers appropriately operationalizing the word unit (Gardner 2007). Further, when corpus-based vocabulary findings are used to inform or support language acquisition, there is the additional concern of whether researcher-based conceptualizations of a word, the criteria used to group and count words, matches the psychological realities of a word, the knowledge of words possessed by the target language users. Thus, learners’ ability to comprehend inflectional and derivational forms requires direct investigation. In L2 lexical and reading research, the most commonly adopted word counting unit is the word family, which consists of the base word and inflected and derived forms from Levels 2–6 of Bauer and Nation’s (1993) affix criteria (hereafter referred to as WF6). Bauer and Nation’s criteria and first language (L1) studies (Tyler and Nagy 1989; Nagy et al. 1993) have been used to justify the use of WF6 as a general word counting unit in both L1 and L2 settings for word list creation (Nation 2006a), vocabulary size tests (i.e., Nation and Beglar 2007), vocabulary levels tests (i.e., McLean and Kramer 2015), and lexical research (Nation 2006b; 2014). Bauer and Nation’s criteria by which English inflectional and derivational affixes were arranged into a graded set of seven levels were created to guide teaching and learning, and standardize vocabulary load and size research. Level ordering divides affixes into a number of types based on their phonological and morphological behavior to determine the level at which a particular affix should be placed. Frequency: The frequency with which an affix occurs. Productivity: The probability that the affix will be used to form new words. Predictability: The degree of predictability of the meaning of the affix. Written base form regularity: The predictability of changes in the written form of the base when the affix is added. Spoken base form regularity: The amount of change in the spoken form of the base when the affix is added. Affix spelling regularity: The predictability of written forms of the affix. Affix form regularity: The predictability of spoken forms of the affix. Regularity of function: The degree to which the affix attaches to a base of known form-class and produces a word of known form-class. Bauer and Nation’s (1993) explanations of a word family for receptive purposes and the operationalization of the word family in research have led to the word family not consisting of a number of possible levels, but commonly being operationalized as WF6 in studies of receptive language use. Bauer and Nation first defined a word family as ‘a base word and all its derived and inflected forms that can be understood by a learner without having to learn each form separately’ (p. 253). Later, Bauer and Nation (1993) stated that ‘watch, watches, watched, and watching may all be members of the same word family for a learner with a command of the inflectional suffixes of English. As a learner's knowledge of affixation develops, the size of the word family increases’ (p. 253). This explanation narrows the word family to a single unit that differs over time and between learners, and thus inhibits the comparison of learners within a single study, and the comparison of research findings between studies. Contrary to this temporal view of the word family that limits the definition of the word family to only forms which are known by a learner is Bauer and Nation’s statement that, ‘The important principle behind the idea of a word family is that once the base word or even a derived word is known, the recognition of other members of the family requires little or no extra effort’ (p. 253). Bauer and Nation provided an example of this principle and stated that once the meaning of marmelize is explained, readers ‘will easily understand He fully deserved his marmelization, which contains a new word-form marmelization’ (p. 253). While this might be true for some learners, less proficient learners might not understand that marmelization is a noun, and might not recognize the meaning of this new word-form. The British National Corpus (BNC) word list (Nation 2006a) and the British National Corpus/Corpus of Contemporary American English (BNC/COCA) word list (Nation 2012), which both utilize WF6, have contributed to the word family commonly being operationalized as WF6 in L2 lexical and reading research. While Nation (2006a) stated that WF6 is appropriate for advanced learners without defining advanced learners, Nation (2012: 3) simply stated that ‘The word family lists group items together that would be perceived as the same words for the receptive skills of listening and reading’. Nation (2014) described the use of WF6 as conservative, when describing the number of words learners with a vocabulary level as low as 1,000 WF6 need to know to comprehend materials and incidentally learn new words from context. A circular argument that WF6 is an appropriate general word counting unit because Bauer and Nation state so is present in the description of the increasingly used Vocabulary Size Test (VST) (Nation and Beglar 2007: 10): ‘all of the family members at Level 6 meet the criteria of regularity, frequency, productivity, and predictability used in developing the word family levels’. Macalister and Webb (2013: 853) stated that the WF6 unit was used in their study, as the study was ‘on receptive vocabulary (comprehension), and for this the word family is “the most sensible unit” (Nation and Webb 2011: 136), as it assumes that a reader who knows one or two members of the word family should be able to recognize and understand other members of the family’. In other cases, the use of WF6 is based on the view that knowing one member of a word family is likely to facilitate recognition of the other members, despite no empirical evidence supporting this. ‘It is important to note that using word families as the unit of counting means that there is the assumption that if the head word is known, all members of the word family will also be known. There is a strong argument that this is true for receptive knowledge of word families because knowing one member of a word family is likely to facilitate recognition of other members’ (Webb 2010: 507). A further issue with the adoption of WF6 in Japanese EFL settings is that evidence for its adoption comes from L1 research and not L2 research. Evidence that WF6 is a psychologically real unit comes from L1 investigations (Tyler and Nagy 1989; Nagy et al. 1993). However, these studies describe incomplete knowledge of derivational forms among high-school students in their L1. Fourth graders were able to recognize familiar L1 English stems in derivatives, and only eighth graders possessed knowledge of the syntactic properties of derivational suffixes (Tyler and Nagy 1989). Furthermore, Nagy et al. (1993) described significant development in suffix knowledge of L1 English learners continuing between the fourth grade and high school, and then even through high school among some learners. Thus, L1 studies present a mixed case for the use of WF6 in L2 English settings, unless we assume that most L2 English learners’ knowledge of derivational forms is greater than L1 English high school students. If the use of WF6 overestimates EFL learners’ knowledge of inflectional and derivational forms, the overestimation will be greatest among low-proficiency learners, as a significant positive correlation has been reported between vocabulary size and affix knowledge (Schmitt and Meara 1997; Mochizuki and Aizawa 2000). This is critical, as the highest frequency words possess the largest number of inflectional and derivational forms. For instance, Brown (2017) in an analysis of the first five 1,000-WF6 bands of the BNC lists shows that the number of word forms among the first, second, third, fourth, and fifth 1,000 words of the BNC lists are 6.348, 5.593, 4.517, 4.288, and 3.992, respectively. Further, Brown shows that texts with 95 per cent coverage on the basis of WF6 have only 82.3, 86.6, 89.5, and 91.2 per cent coverage when Levels 2-, 3-, 4-, and 5-word family forms are included, respectively.1 Similarly, texts with 98 per cent coverage on the basis of WF6 only have 84.9, 89.3, 92.4, and 94.1 per cent coverage if Levels 2-, 3-, 4-, and 5-word family forms are included, respectively. Hu and Nation (2000) and Schmitt, Jiang, and Grabe (2011) stated that such drops in coverage inhibit reading comprehension. An alternative to WF6 is the lemma or flemma. The lemma consists of a headword and its inflected, irregular, and reduced forms (n't) that are of the same part of speech (Francis and Kucˇera 1982). The English inflected forms are plural, third person singular present tense, past tense, past participle, -ing, comparative, superlative, and possessive forms. While the word family develop consists of develop, develops, developed, developing, developable, development, developments, developmental, undevelopable, developer, developers, undeveloped, developmental, developmentally, developmentwise, semideveloped, antidevelopment, redevelop, and predevelopment, the lemma develop only consists of develop, develops, developed, and developing. Swenson and West’s (1934) concept of learning burden underpins the lemma, positing that the learning burden or difficultly in inferring the meaning of inflected forms from the base form is minor. The flemma, unlike the lemma, groups identical forms of different parts of speech. Thus, the verb developed and the adjective developed are different lemmas, but members of the same flemma (Pinchbeck, 2014). Schmitt (2010) argued that the lemma might be the best general unit of counting for four reasons. First, what forms are included within a lemma is straightforward relative to the word family, facilitating the easy comprehension of research studies. Further, opinions differ on what to do with hyphenated forms and rare forms such as -wise or -like. Second, the comparatively straightforward lemma makes the replication and comparison of studies more feasible. Third, the lemma is a pragmatic compromise when counting both receptive and productive vocabulary use, facilitating the comparison of receptive and productive studies. No researchers have suggested the adoption of WF6 when measuring productive lexical knowledge. Finally, Schmitt argued that, relative to WF6, the lemma provides more realistic estimates of the large number of words necessary to function in English. This notion is supported by existing L2 research (Schmitt and Meara 1997; Mochizuki and Aizawa 2000; Ward and Chuenjundaeng 2009; Sasao and Webb 2015) which indicates that Thai and Japanese EFL learners comprehend a limited number of WF6 derivational forms. Evidence that the flemma is a more appropriate general written receptive word counting unit than WF6 with Japanese EFL learners comes from four studies. Schmitt and Meara (1997) reported that Japanese university and high school participants demonstrated meaning-recognition knowledge of 52.85 per cent of affixes at Levels 2–6 of Bauer and Nation’s (1993) affix criteria, including -ed (92 per cent), -able (30 per cent), -ence (58 per cent), and -al (10 per cent). Mochizuki and Aizawa (2000), using pseudoword base forms and multiple-choice tests, found that Japanese university and high school participants knew (meaning recognition) on average 7.24 (56 per cent) of 13 prefixes, and 10.70 (67 per cent) of 16 suffixes. Ward and Chuenjundaeng (2009) investigated 257 Thai learners’ ability to recall the meaning of both the base and derived forms of 32 target words. Among intermediate learners (n = 167) only 41.8, 33.0, 40.6, and 59.5 per cent demonstrated knowledge (meaning-recall) of the derived -tion, -er, -ment, and -ity forms, respectively. Among more proficient medical students (n = 90) only 69.2, 69.5, 56.0, and 42.8 per cent demonstrated knowledge (meaning-recall) of the derived -tion, -er, -ment, and -ity forms, respectively. Finally, Sasao and Webb (2015) investigated 1,348 participants’ meaning-recognition knowledge of 118 affixes. The results from multiple-choice tests revealed the relative difficulty of 118 affixes and indicated that the participants’ knowledge (form, meaning, and use) of the 118 affixes was far from complete. These four studies indicate that their participants had limited receptive knowledge of inflectional and derivational forms. However, the shared and/or individual limitations of these studies prevent valid conclusions being made regarding the degree to which Japanese learners of different proficiencies are able to comprehend inflectional and derivational forms. First, only Sasao and Webb (2015) stated the reliability of data collected. Second, Schmitt and Meara (1997) and Ward and Chuenjundaeng (2009) only tested knowledge of suffixes, and not prefixes. Third, none of the four studies tested knowledge of words with multiple affixes. For example, the word reusable consists of the base form, use, and the affixes re- and -able. Brown (2017) found that 26.8 per cent of the total number of word forms from the first 5,000 WF6 of the BNC contained two or more affixes. Fourth, the use of multiple-choice instruments (Schmitt and Meara 1997; Mochizuki and Aizawa 2000; Sasao and Webb 2015) overestimates learners’ knowledge and limits the ecological validity of inferences. Sasao and Webb stated that guessing with multiple-choice instruments overestimates learners’ knowledge and so corrected for lucky guessing (unexpected success by low-ability respondents). However, this procedure fails to distinguish between idiosyncratic vocabulary knowledge and guessing, and the accuracy of this correction is unknown. Thus, while Mochizuki and Aizawa, Sasao and Webb, and Schmitt and Meara made major contributions to our understanding of learners’ knowledge of affixes relative to other affixes through the use of multiple-choice tests, the degree to which the data collected in these studies accurately represent the learners’ actual ability to comprehend inflectional and derivational forms is unknown. Furthermore, the ecological validity of using a multiple-choice test (meaning recognition) to operationalize students ability to recall the meaning of derived forms as when reading has been questioned by Gyllstad, Vilkaite, and Schmitt (2015). This is because meaning recognition requires a lower threshold of lexical knowledge than meaning recall (Laufer and Goldstein 2004; Schmitt 2010; Nation 2013; Nation and Webb 2011), and when reading, the meanings of words are recalled, and not recognized from four possible options. Finally, Mochizuki and Aizawa (2000) and Sasao and Webb (2015) also investigated their participants’ metalinguistic knowledge of affixes (i.e., the part of speech of affixes) and not their ability to comprehend inflectional and derivational forms. The limitations of the existing research fuel continued calls for research that directly investigates the appropriateness of WF6: ‘we agree on the need for further research on the actual performance of second language learners when they encounter members of a word family’ (Macalister and Webb 2013: 853). Thus, this study investigates Japanese EFL learners’ ability to comprehend (produce the L1 meaning from the L2 form) inflectional and derivational WF6 forms consisting of a single affix or multiple affixes, and presents the following three research questions. RQ1 Is there a significant difference in Japanese EFL learners’ ability to comprehend the base form, inflectional forms, and derivational forms of the same word family? RQ2 Is there a significant difference in Japanese EFL learners’ ability to comprehend the base form and inflectional forms of the same word family? RQ3 What inflectional and derivational forms does the flemma overestimate or underestimate participants’ ability to comprehend? These research questions will be addressed by first investigating the participants as a single group, and then as three groups classified by lexical proficiency. METHODOLOGY Participants The participants (N = 279), aged between 18 and 24 years, were Japanese EFL university students who had studied English formally for six years in secondary school. In total, 185 participants (87 females and 98 males) were first-year university students enrolled in 3 hours per week of English language courses. They belonged to their institution’s intermediate English stream. In total, 50 participants (4 females and 46 males) were first-year university students enrolled in 3 hours per week of English language courses. They belonged to their institution’s advanced English stream. In total, 21 participants (16 females and 5 males) were second-year students who had elected to study in upper advanced English classes, and were enrolled in 3.0–4.5 hours of English language courses per week. In total, 23 participants, 10 third-year (9 females and 1 male) and 13 fourth-year (12 females and 1 male) students, were English majors. An additional 19 students failed to complete both instruments used in this study; thus, their data were not included. The participants came from intact classes taught by the researcher, and they gave written consent for the data in this research to be used. The majority of the participants had not completed a standardized English proficiency test. However, the participants’ New Vocabulary Levels Test (NVLT) scores (McLean and Kramer 2015) provide an indication of the participants’ written receptive meaning-recognition vocabulary knowledge by which they were separated into three groups. The beginner group (n = 85) consists of participants who failed to score 22 or more out of 24 on the second 1,000-word band of the NVLT. The intermediate group (n = 177) scored 22 or more on each of the second and third 1,000-word bands, but failed to score 22 or more on the fourth 1,000-word band. The advanced group (n = 17) scored 22 or more on the fourth and/or the fifth 1,000-word bands. While a cutoff score of 24 out of 24 would have been preferable for mastery, such a strict cutoff score would not allow for measurement error. A one-way analysis of variance (ANOVA) with groups as the independent variable and NVLT scores as the dependent variable found that the NVLT scores among these three groups differed significantly, F(2, 276) = 126.253, p = 0.000, with significant differences (p < 0.001) in NVLT scores between the advanced group and the intermediate group, the advanced group and the beginner group, and the intermediate group and the beginner group. Only the advanced-group participants had all completed a standardized test, and had shared learning experiences. They were all third-year or fourth-year English majors who had spent a year studying abroad in English-medium classes at universities in native English-speaking nations, and possessed English proficiencies very rarely met even at graduate school level in Japan, with a mean TOEIC score of 872. These participants had correctly answered at least 22 out of 24 multiple-choice vocabulary base word items from the 4,000 and/or 5,000 BNC/COCA WF6 bands. Webb and Sasao (2013) refer to the first 5,000 words of the BNC/COCA as the greatest range in vocabulary learning for the majority of L2 learners. Instruments Inflectional and derivational forms comprehension test This study sets out to measure the participants’ ability to comprehend the inflectional and derivational forms of known base words. Selecting target base forms too difficult for the participants would decrease the amount of data available for analysis. Thus, knowledge of 20 high-frequency words from the first 2,000 base words of the BNC by 17 low-proficiency participants was investigated. Twelve words (use, move, collect, center, teach, accept, maintain, develop, standard, circle, adjust, and publish) were known by all participants. The 12 words possess too many inflectional and derivational forms to be tested in their entirety. Thus, 87 inflectional and derivational forms were selected with the following criteria. Inflectional and derivational forms are from Levels 2–6 of Bauer and Nation’s (1993) criteria. Target forms include affixes that can be attached to free morphemes. However, some items consisted of multiple affixes that can be attached to free morphemes. For example, the form standardization consists of the affixes -ize and -ation. The derivational forms include affixes present in Nation’s (2001) list of derivational affixes for learners. Where possible, the same affix is present in target words from different word families to investigate possible variations in participants’ ability to comprehend different words containing the same affixes. For example, -ment was present in the items movement, adjustment, and development. The test did not include inflectional forms marking plurals (-s) and the third person (-s, -es) because Japanese does not mark these forms, making the testing of them in a translation test problematic, and because they have minimal impact on reading comprehension. The resulting 100-item comprehension (L2 form to L1 meaning translation) test (see Supplementary Materials) presented the target words in a context sentence using vocabulary from the first 2,000 WF6 of the BNC/COCA (Nation 2012), or other words (including loanwords) that piloting indicated that the participants would comprehend (panda, vocabulary, text, snack, network, and appropriate). Here is an example of one item: The computer is now usable = _____________. Instructions for the tests were presented in the participants’ L1, Japanese. Instructions accompanying the test were as follows.2Please translate the underlined word into Japanese. The answer can be expressed as a single word or multiple words. Please pay attention to the form of the underlined word. Please ensure that your writing is easy to read. The items belonging to a single WF6 were presented in sets. However, the order in which target WF6 members were presented differed to prevent the participants from guessing the meaning of the target form based on their order. For example, the past tense form did not always follow the base word (see Supplementary materials online). The test was completed within 30 minutes in class and was administered by the participants’ teacher and author of this article. Separating students into lexical proficiency groups The NVLT (McLean and Kramer 2015) consists of 24 multiple-choice items per 1,000-BNC/COCA WF6 band for the first five 1,000-word bands. The NVLT was administered by the author one week after the inflectional and derivational forms comprehension test, and the participants completed the NVLT within 30 minutes. Examinees select the Japanese response with the closest meaning to the target word from four options. A bilingual Japanese and not monolingual English version of the NVLT was given, as bilingual multiple-choice tests increase the unidimensionality of data collected and are completed more quickly (McLean et al. 2016). The test instructions were written in the examinees’ native language (Japanese) to support rapid and accurate comprehension. Examinees were instructed to skip questions they had absolutely no idea of. An example item is shown below. The translations in parentheses were not visible to examinees. School: This is a big school. 1 銀行 (bank) 海の動物 (sea animal) 学校 (school) 家 (house) Scoring The NVLT and the inflectional and derivational forms comprehension test answers were marked dichotomously. The NVLT data yielded a KR20 reliability estimate of 0.88. One Japanese native speaker, a teacher of English, marked all of the inflectional and derivational forms comprehension tests. Two further native speakers of Japanese, teachers of English, marked a randomly selected subset of 20 per cent of the inflectional and derivational forms comprehension tests. Using Kappa analysis, the interrater reliability values established among the three markers were all ≥0.91. The KR20 reliability estimate of the entire inflectional and derivational comprehension test data set marked by the main marker was 0.93, with estimates of 0.97 and 0.97 from data marked by the second and third marker, respectively. If the L1 Japanese meaning produced by the participants demonstrated accurate knowledge of the target word including the meaning of the target form’s affix(es), markers marked the answer as correct. Importantly, when marking the translations of the base forms, the answer was marked as correct if a participant answered with the noun form rather than the intended verb form for three reasons. First, the majority of vocabulary size tests and vocabulary levels tests are multiple-choice or Yes/No tests. The Yes/No tests (Meara and Buxton 1987) and Vocabulary Levels Test (VLT) (Schmitt et al. 2001) present target words in a decontextualized form. Thus, examinees who do not know that center can be a verb, but know the meaning of the noun center will state that they know the word center on a Yes/No test, or they can correctly select a multiple-choice answer on the VLT. Similarly, multiple-choice tests such as the VST and NVLT commonly present items in a decontextualized form and then in a sentence (e.g. see: They saw it). Third, and most importantly, word family and flemma-based lists group different parts of speech. Thus, vocabulary lists, tests, lexical profiling tools, and research based on them do not distinguish between parts of speech when estimating lexical knowledge. As a result, for the purpose of ecological validity, these answers were marked as correct when participants answered with a noun form instead of a verb form. Data analysis The following analysis is of data collected from only inflectional form items and derivational form items that the participants first correctly translated the associated L2 English base form into the L1 Japanese meaning. Thus, if a participant failed to comprehend the meaning of publish, that participant’s responses to the base form, inflectional and derivational members of the publish WF6 were excluded from the analysis. If the same participant correctly produced the L2 meaning of use, that participant’s responses to use WF6 members were included in the analysis. This step was taken to ensure that the data represented the participants’ ability to comprehend inflectional and derivational forms and not their ability to comprehend base forms. Of the 279 participants, the following number incorrectly answered the base form items: use, move, collect (2), circle (6), center (7), accept (24), maintain, develop (11), standard (12), adjust (16), and publish (10). Research questions 1 and 2 were investigated using Cochran’s Q analysis, a nonparametric alternative to an ANOVA which allows for the analysis of multiple repeated measures in the form of dichotomous data. In this study the participants’ performance on the base word and each inflectional and derivational form of that base word were treated as repeated measures. Thus, a significant difference yielded from the dichotomous scores among the base word use, and the tested inflectional forms (used, using, and have used) and tested derivational forms (useful, useless, reuse, reusable, unusable, usage, and user) would indicate a significant difference in the participants’ ability to comprehend the tested forms. Alpha values used in this study for each individual Cochran’s Q analysis were set at 0.001. Research question 3 was investigated using McNemar chi-square analysis, which can be conceptualized as post hoc ANOVA analysis, and is a nonparametric analysis that allows for the analysis of two repeated measures with dichotomous data. To avoid a type 1 error, the alpha values for the McNemar chi-square analysis are dependent on the number of comparisons made between the base word and other members of the same word family. The alpha value was arrived at by dividing 0.05 by the number of comparisons. Cochran’s Q and McNemar chi-square analysis have two assumptions. The first assumption, that data scores are independent of each other, is met. Second, accurate analysis is possible when sample sizes are large, or above 30 participants for Cochran’s Q analysis, while a sample of 26 is sufficient for accurate McNemar chi-square analysis (Green and Salkind 2013). The small number of advanced-group participants (n = 17), and, in contrast, the large size of the entire data set (N = 279), which facilitates small differences in the participants’ ability to comprehend the base form and associated forms yielding statistical significance, means that inferences should be supported by effect-size values. This study adopts Brown and Coombe’s (2015) recommendations for small, medium, and large eta squared (η2) effect sizes of 0.02, 0.13, and 0.26 for Cochran’s Q analysis. The effect-size criteria for the McNemar (φ) analysis are 0.1 (small), 0.3 (medium), 0.5 (large), and 0.8 (very large) (Rovai et al. 2013). RESULTS Table 1 shows the number of participants who correctly answered the base form, and then associated inflectional and derivational forms for each of the 12 target words. Table 1: Number of participants who correctly answered base forms, and inflectional and derivational forms Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle  Adjust  Publish  Base form  277  277  277  272  279  255  268  268  267  273  263  269  -ed  275  277  276  251  275  252  265  261    263  260  266  -ing  276  274  271  240  276  250  261  261    255  258  265  -ing adjective                245          have-ed  270  271  270  237  272  248  260  261    256  260  265  -er  271        276      225      236  265  -able  221    222                194    -less  164                        un-able                150        158  -erly          141                -ingly    139                      un-ing            105              un-ed              89      115      non-                  68        -ment    232            206      194    -ful  200                        -ized                  118        -mentally                77          -ability          62  54  65          76  -ally        215                  -ance            159  185            mis-ed          166              112  anti-ment                80          -alized        74                  -age  63                        counter-    44                      sub-                  28        mal-ed                      19    semi-                    18      en-ment                    16      re-  243          190    225    206  203    -tion      234                    re-able  202                        re-ing                        164  -ic        115                  pre-ed          131                -ist        162                  -ization                  109        -ivization      57                    Noun form                    273      Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle  Adjust  Publish  Base form  277  277  277  272  279  255  268  268  267  273  263  269  -ed  275  277  276  251  275  252  265  261    263  260  266  -ing  276  274  271  240  276  250  261  261    255  258  265  -ing adjective                245          have-ed  270  271  270  237  272  248  260  261    256  260  265  -er  271        276      225      236  265  -able  221    222                194    -less  164                        un-able                150        158  -erly          141                -ingly    139                      un-ing            105              un-ed              89      115      non-                  68        -ment    232            206      194    -ful  200                        -ized                  118        -mentally                77          -ability          62  54  65          76  -ally        215                  -ance            159  185            mis-ed          166              112  anti-ment                80          -alized        74                  -age  63                        counter-    44                      sub-                  28        mal-ed                      19    semi-                    18      en-ment                    16      re-  243          190    225    206  203    -tion      234                    re-able  202                        re-ing                        164  -ic        115                  pre-ed          131                -ist        162                  -ization                  109        -ivization      57                    Noun form                    273      Table 1: Number of participants who correctly answered base forms, and inflectional and derivational forms Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle  Adjust  Publish  Base form  277  277  277  272  279  255  268  268  267  273  263  269  -ed  275  277  276  251  275  252  265  261    263  260  266  -ing  276  274  271  240  276  250  261  261    255  258  265  -ing adjective                245          have-ed  270  271  270  237  272  248  260  261    256  260  265  -er  271        276      225      236  265  -able  221    222                194    -less  164                        un-able                150        158  -erly          141                -ingly    139                      un-ing            105              un-ed              89      115      non-                  68        -ment    232            206      194    -ful  200                        -ized                  118        -mentally                77          -ability          62  54  65          76  -ally        215                  -ance            159  185            mis-ed          166              112  anti-ment                80          -alized        74                  -age  63                        counter-    44                      sub-                  28        mal-ed                      19    semi-                    18      en-ment                    16      re-  243          190    225    206  203    -tion      234                    re-able  202                        re-ing                        164  -ic        115                  pre-ed          131                -ist        162                  -ization                  109        -ivization      57                    Noun form                    273      Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle  Adjust  Publish  Base form  277  277  277  272  279  255  268  268  267  273  263  269  -ed  275  277  276  251  275  252  265  261    263  260  266  -ing  276  274  271  240  276  250  261  261    255  258  265  -ing adjective                245          have-ed  270  271  270  237  272  248  260  261    256  260  265  -er  271        276      225      236  265  -able  221    222                194    -less  164                        un-able                150        158  -erly          141                -ingly    139                      un-ing            105              un-ed              89      115      non-                  68        -ment    232            206      194    -ful  200                        -ized                  118        -mentally                77          -ability          62  54  65          76  -ally        215                  -ance            159  185            mis-ed          166              112  anti-ment                80          -alized        74                  -age  63                        counter-    44                      sub-                  28        mal-ed                      19    semi-                    18      en-ment                    16      re-  243          190    225    206  203    -tion      234                    re-able  202                        re-ing                        164  -ic        115                  pre-ed          131                -ist        162                  -ization                  109        -ivization      57                    Noun form                    273      Research question 1 asked if there is a significant difference in Japanese EFL learners’ ability to comprehend the base form, inflectional forms, and derivational forms. Considering the very high threshold of 98 per cent knowledge of tokens within a text necessary for reading comprehension, the overestimation of learners’ ability to comprehend inflectional and derivational forms is a greater concern than its underestimation. Thus, for WF6 to be recommended as an appropriate general counting unit, a significant difference between the participants’ ability to comprehend the base form, inflectional forms, and derivational forms should not be present. The absence of a significant difference would suggest that the participants’ ability to comprehend the base word is similar to that of the tested inflectional and derivational forms of that base word, and would support the assumption made when using WF6 that comprehension of the base word is evidence of being able to comprehend all other inflectional and derivational forms belonging to that WF6. This is why, when creating WF6 word lists, the presence of derivational forms, for example reusable, is added to the frequency of the use WF6 in that corpus. Likewise, test-takers are accredited with knowledge of unseeingly, if they correctly demonstrate knowledge of see on the WF6-based VST. Or when the lexical load of a text is estimated with a WF6-based lexical profiler, the derived forms of centralization and centric among other derivational forms of center are counted as high-frequency words from the first 1,000 WF6. In contrast, if a significant difference is present, it indicates that the participants’ ability to comprehend a base word is significantly different to that of tested inflectional and derivational forms of that base word, and would not support the assumption made when using WF6. The Cochran’s Q analysis indicated a significant difference (p < 0.001) among the number of correct responses to all 12 base words, and correct responses to associated inflectional and derivational forms (Table 2). The large effect sizes (η2 = 0.36–0.71) of these differences are presented in Table 2. This pattern of significant differences among the number of correct responses to the base word and correct responses to inflectional and derivational forms of the same WF6 is present across all 12 of the target word families among the beginner-group (η2 = 0.44–0.71), intermediate-group (η2 = 0.34–0.73), and advanced-group (η2 = 0.23–0.84) participants, with the exception of the advanced-group participants’ performance on the publish WF6 forms (p = 0.001, η2 = 0.19). The statistically significant discrepancies between the participants’ ability to comprehend the base word and other members of the same WF6 accompanied by large effect sizes (Tables 1 and 2) strongly indicate that WF6 overestimates the participants’ ability to comprehend inflectional and derivational forms. Thus, considering the high threshold of 98 per cent knowledge of tokens within a text necessary for unassisted reading comprehension, WF6 is not the most appropriate general written receptive word counting unit for Japanese EFL learners. Table 2: The significance and effect size of differences in the number of participants who comprehend base forms and the number of participants who comprehend associated WF6 members WF6  All participants  Beginner group  Medium group  Advanced group  Use  N = 277, p = 0.000, η2 = 0.39  n = 84, p = 0.000, η2 = 0.46  n = 176, p = 0.000, η2 = 0.40  n = 17, p = 0.000, η2 = 0.28  Move  N = 277, p = 0.000, η2 = 0.71  n = 84, p = 0.000, η2 = 0.74  n = 176, p = 0.000, η2 = 0.73  n = 17, p = 0.000, η2 = 0.63  Collect  N = 277, p = 0.000, η2 = 0.55  n = 84, p = 0.000, η2 = 0.55  n = 176, p = 0.000, η2 = 0.58  n = 17, p = 0.000, η2 = 0.29  Center  N = 272, p = 0.000, η2 = 0.36  n = 82, p = 0.000, η2 = 0.44  n = 173, p = 0.000, η2 = 0.34  n = 17, p = 0.000, η2 = 0.27  Teach  N = 279, p = 0.000, η2 = 0.48  n = 85, p = 0.000, η2 = 0.56  n = 177, p = 0.000, η2 = 0.48  n = 17, p = 0.000, η2 = 0.24  Accept  N = 255, p = 0.000, η2 = 0.47  n = 70, p = 0.000, η2 = 0.57  n = 168, p = 0.000, η2 = 0.47  n = 17, p = 0.000, η2 = 0.23  Maintain  N = 268, p = 0.000, η2 = 0.55  n = 74, p = 0.000, η2 = 0.67  n = 177, p = 0.000, η2 = 0.54  n = 17, p = 0.000, η2 = 0.30  Develop  N = 268, p = 0.000, η2 = 0.41  n = 77, p = 0.000, η2 = 0.46  n = 174, p = 0.000, η2 = 0.41  n = 17, p = 0.000, η2 = 0.27  Standard  N = 267, p = 0.000, η2 = 0.46  n = 78, p = 0.000, η2 = 0.55  n = 172, p = 0.000, η2 = 0.46  n = 17, p = 0.000, η2 = 0.41  Circle  N = 273, p = 0.000, η2 = 0.64  n = 83, p = 0.000, η2 = 0.71  n = 173, p = 0.000, η2 = 0.70  n = 17, p = 0.000, η2 = 0.84  Adjust  N = 263, p = 0.000, η2 = 0.54  n = 75, p = 0.000, η2 = 0.50  n = 171, p = 0.000, η2 = 0.55  n = 17, p = 0.000, η2 = 0.72  Publish  N = 269, p = 0.000, η2 = 0.46  n = 81, p = 0.000, η2 = 0.59  n = 171, p = 0.000, η2 = 0.44  n = 17, p = 0.001, η2 = 0.19  WF6  All participants  Beginner group  Medium group  Advanced group  Use  N = 277, p = 0.000, η2 = 0.39  n = 84, p = 0.000, η2 = 0.46  n = 176, p = 0.000, η2 = 0.40  n = 17, p = 0.000, η2 = 0.28  Move  N = 277, p = 0.000, η2 = 0.71  n = 84, p = 0.000, η2 = 0.74  n = 176, p = 0.000, η2 = 0.73  n = 17, p = 0.000, η2 = 0.63  Collect  N = 277, p = 0.000, η2 = 0.55  n = 84, p = 0.000, η2 = 0.55  n = 176, p = 0.000, η2 = 0.58  n = 17, p = 0.000, η2 = 0.29  Center  N = 272, p = 0.000, η2 = 0.36  n = 82, p = 0.000, η2 = 0.44  n = 173, p = 0.000, η2 = 0.34  n = 17, p = 0.000, η2 = 0.27  Teach  N = 279, p = 0.000, η2 = 0.48  n = 85, p = 0.000, η2 = 0.56  n = 177, p = 0.000, η2 = 0.48  n = 17, p = 0.000, η2 = 0.24  Accept  N = 255, p = 0.000, η2 = 0.47  n = 70, p = 0.000, η2 = 0.57  n = 168, p = 0.000, η2 = 0.47  n = 17, p = 0.000, η2 = 0.23  Maintain  N = 268, p = 0.000, η2 = 0.55  n = 74, p = 0.000, η2 = 0.67  n = 177, p = 0.000, η2 = 0.54  n = 17, p = 0.000, η2 = 0.30  Develop  N = 268, p = 0.000, η2 = 0.41  n = 77, p = 0.000, η2 = 0.46  n = 174, p = 0.000, η2 = 0.41  n = 17, p = 0.000, η2 = 0.27  Standard  N = 267, p = 0.000, η2 = 0.46  n = 78, p = 0.000, η2 = 0.55  n = 172, p = 0.000, η2 = 0.46  n = 17, p = 0.000, η2 = 0.41  Circle  N = 273, p = 0.000, η2 = 0.64  n = 83, p = 0.000, η2 = 0.71  n = 173, p = 0.000, η2 = 0.70  n = 17, p = 0.000, η2 = 0.84  Adjust  N = 263, p = 0.000, η2 = 0.54  n = 75, p = 0.000, η2 = 0.50  n = 171, p = 0.000, η2 = 0.55  n = 17, p = 0.000, η2 = 0.72  Publish  N = 269, p = 0.000, η2 = 0.46  n = 81, p = 0.000, η2 = 0.59  n = 171, p = 0.000, η2 = 0.44  n = 17, p = 0.001, η2 = 0.19  Table 2: The significance and effect size of differences in the number of participants who comprehend base forms and the number of participants who comprehend associated WF6 members WF6  All participants  Beginner group  Medium group  Advanced group  Use  N = 277, p = 0.000, η2 = 0.39  n = 84, p = 0.000, η2 = 0.46  n = 176, p = 0.000, η2 = 0.40  n = 17, p = 0.000, η2 = 0.28  Move  N = 277, p = 0.000, η2 = 0.71  n = 84, p = 0.000, η2 = 0.74  n = 176, p = 0.000, η2 = 0.73  n = 17, p = 0.000, η2 = 0.63  Collect  N = 277, p = 0.000, η2 = 0.55  n = 84, p = 0.000, η2 = 0.55  n = 176, p = 0.000, η2 = 0.58  n = 17, p = 0.000, η2 = 0.29  Center  N = 272, p = 0.000, η2 = 0.36  n = 82, p = 0.000, η2 = 0.44  n = 173, p = 0.000, η2 = 0.34  n = 17, p = 0.000, η2 = 0.27  Teach  N = 279, p = 0.000, η2 = 0.48  n = 85, p = 0.000, η2 = 0.56  n = 177, p = 0.000, η2 = 0.48  n = 17, p = 0.000, η2 = 0.24  Accept  N = 255, p = 0.000, η2 = 0.47  n = 70, p = 0.000, η2 = 0.57  n = 168, p = 0.000, η2 = 0.47  n = 17, p = 0.000, η2 = 0.23  Maintain  N = 268, p = 0.000, η2 = 0.55  n = 74, p = 0.000, η2 = 0.67  n = 177, p = 0.000, η2 = 0.54  n = 17, p = 0.000, η2 = 0.30  Develop  N = 268, p = 0.000, η2 = 0.41  n = 77, p = 0.000, η2 = 0.46  n = 174, p = 0.000, η2 = 0.41  n = 17, p = 0.000, η2 = 0.27  Standard  N = 267, p = 0.000, η2 = 0.46  n = 78, p = 0.000, η2 = 0.55  n = 172, p = 0.000, η2 = 0.46  n = 17, p = 0.000, η2 = 0.41  Circle  N = 273, p = 0.000, η2 = 0.64  n = 83, p = 0.000, η2 = 0.71  n = 173, p = 0.000, η2 = 0.70  n = 17, p = 0.000, η2 = 0.84  Adjust  N = 263, p = 0.000, η2 = 0.54  n = 75, p = 0.000, η2 = 0.50  n = 171, p = 0.000, η2 = 0.55  n = 17, p = 0.000, η2 = 0.72  Publish  N = 269, p = 0.000, η2 = 0.46  n = 81, p = 0.000, η2 = 0.59  n = 171, p = 0.000, η2 = 0.44  n = 17, p = 0.001, η2 = 0.19  WF6  All participants  Beginner group  Medium group  Advanced group  Use  N = 277, p = 0.000, η2 = 0.39  n = 84, p = 0.000, η2 = 0.46  n = 176, p = 0.000, η2 = 0.40  n = 17, p = 0.000, η2 = 0.28  Move  N = 277, p = 0.000, η2 = 0.71  n = 84, p = 0.000, η2 = 0.74  n = 176, p = 0.000, η2 = 0.73  n = 17, p = 0.000, η2 = 0.63  Collect  N = 277, p = 0.000, η2 = 0.55  n = 84, p = 0.000, η2 = 0.55  n = 176, p = 0.000, η2 = 0.58  n = 17, p = 0.000, η2 = 0.29  Center  N = 272, p = 0.000, η2 = 0.36  n = 82, p = 0.000, η2 = 0.44  n = 173, p = 0.000, η2 = 0.34  n = 17, p = 0.000, η2 = 0.27  Teach  N = 279, p = 0.000, η2 = 0.48  n = 85, p = 0.000, η2 = 0.56  n = 177, p = 0.000, η2 = 0.48  n = 17, p = 0.000, η2 = 0.24  Accept  N = 255, p = 0.000, η2 = 0.47  n = 70, p = 0.000, η2 = 0.57  n = 168, p = 0.000, η2 = 0.47  n = 17, p = 0.000, η2 = 0.23  Maintain  N = 268, p = 0.000, η2 = 0.55  n = 74, p = 0.000, η2 = 0.67  n = 177, p = 0.000, η2 = 0.54  n = 17, p = 0.000, η2 = 0.30  Develop  N = 268, p = 0.000, η2 = 0.41  n = 77, p = 0.000, η2 = 0.46  n = 174, p = 0.000, η2 = 0.41  n = 17, p = 0.000, η2 = 0.27  Standard  N = 267, p = 0.000, η2 = 0.46  n = 78, p = 0.000, η2 = 0.55  n = 172, p = 0.000, η2 = 0.46  n = 17, p = 0.000, η2 = 0.41  Circle  N = 273, p = 0.000, η2 = 0.64  n = 83, p = 0.000, η2 = 0.71  n = 173, p = 0.000, η2 = 0.70  n = 17, p = 0.000, η2 = 0.84  Adjust  N = 263, p = 0.000, η2 = 0.54  n = 75, p = 0.000, η2 = 0.50  n = 171, p = 0.000, η2 = 0.55  n = 17, p = 0.000, η2 = 0.72  Publish  N = 269, p = 0.000, η2 = 0.46  n = 81, p = 0.000, η2 = 0.59  n = 171, p = 0.000, η2 = 0.44  n = 17, p = 0.001, η2 = 0.19  For the flemma to be recommended, a significant difference between the participants’ ability to comprehend the base form and inflectional forms should not be present. Thus, research question 2 asks if there is a significant difference in Japanese EFL learners’ ability to comprehend the base form and associated inflectional forms, and is investigated using Cochran’s Q analysis. Significant differences (p < 0.001) accompanied by small effect sizes were only found among the number of correct responses to three base words (center, η2 = 0.09; develop, η2 = 0.02; circle, and η2 = 0.05) and associated inflectional forms (Table 3). However, a nonsignificant difference is present for eight (use, move, collect, teach, accept, maintain, adjust, and publish) of the 11 flemmas.3 Table 3: The significance and effect size of differences in the number of participants who comprehend base forms and the number of participants who comprehend associated flemma members Flemma  All participants  Beginner group  Intermediate group  Advanced group  Use  N = 277, p = 0.006, η2 = 0.01  n = 84, p = 0.137, η2 = 0.02  n = 176, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Move  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.014, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Collect  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.013, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Center  N = 272, p = 0.000, η2 = 0.09  n = 82, p = 0.000, η2 = 0.1  n = 173, p = 0.000, η2 = 0.09  n = 17, p = 1, η2 = 0.00  Teach  N = 279, p = 0.019, η2 = 0.01  n = 85, p = 0.084, η2 = 0.02  n = 177, p = 0.194, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Accept  N = 255, p = 0.008, η2 = 0.02  n = 70, p = 0.300, η2 = 0.02  n = 168, p = 0.034, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Maintain  N = 268, p = 0.003, η2 = 0.02  n = 74, p = 0.004, η2 = 0.06  n = 177, p = 0.340, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Develop  N = 268, p = 0.000, η2 = 0.03  n = 77, p = 0.007, η2 = 0.05  n = 174, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Standard          Circle  N = 273, p = 0.000, η2 = 0.05  n = 83, p = 0.000, η2 = 0.09  n = 173, p = 0.000, η2 = 0.03  n = 17, p = 1, η2 = 0.00  Adjust  N = 263, p = 0.017, η2 = 0.01  n = 75, p = 0.029, η2 = 0.04  n = 171, p = 0.112, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Publish  N = 269, p = 0.105, η2 = 0.01  n = 81, p = 0.096, η2 = 0.03  n = 171, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Flemma  All participants  Beginner group  Intermediate group  Advanced group  Use  N = 277, p = 0.006, η2 = 0.01  n = 84, p = 0.137, η2 = 0.02  n = 176, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Move  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.014, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Collect  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.013, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Center  N = 272, p = 0.000, η2 = 0.09  n = 82, p = 0.000, η2 = 0.1  n = 173, p = 0.000, η2 = 0.09  n = 17, p = 1, η2 = 0.00  Teach  N = 279, p = 0.019, η2 = 0.01  n = 85, p = 0.084, η2 = 0.02  n = 177, p = 0.194, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Accept  N = 255, p = 0.008, η2 = 0.02  n = 70, p = 0.300, η2 = 0.02  n = 168, p = 0.034, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Maintain  N = 268, p = 0.003, η2 = 0.02  n = 74, p = 0.004, η2 = 0.06  n = 177, p = 0.340, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Develop  N = 268, p = 0.000, η2 = 0.03  n = 77, p = 0.007, η2 = 0.05  n = 174, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Standard          Circle  N = 273, p = 0.000, η2 = 0.05  n = 83, p = 0.000, η2 = 0.09  n = 173, p = 0.000, η2 = 0.03  n = 17, p = 1, η2 = 0.00  Adjust  N = 263, p = 0.017, η2 = 0.01  n = 75, p = 0.029, η2 = 0.04  n = 171, p = 0.112, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Publish  N = 269, p = 0.105, η2 = 0.01  n = 81, p = 0.096, η2 = 0.03  n = 171, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Table 3: The significance and effect size of differences in the number of participants who comprehend base forms and the number of participants who comprehend associated flemma members Flemma  All participants  Beginner group  Intermediate group  Advanced group  Use  N = 277, p = 0.006, η2 = 0.01  n = 84, p = 0.137, η2 = 0.02  n = 176, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Move  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.014, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Collect  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.013, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Center  N = 272, p = 0.000, η2 = 0.09  n = 82, p = 0.000, η2 = 0.1  n = 173, p = 0.000, η2 = 0.09  n = 17, p = 1, η2 = 0.00  Teach  N = 279, p = 0.019, η2 = 0.01  n = 85, p = 0.084, η2 = 0.02  n = 177, p = 0.194, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Accept  N = 255, p = 0.008, η2 = 0.02  n = 70, p = 0.300, η2 = 0.02  n = 168, p = 0.034, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Maintain  N = 268, p = 0.003, η2 = 0.02  n = 74, p = 0.004, η2 = 0.06  n = 177, p = 0.340, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Develop  N = 268, p = 0.000, η2 = 0.03  n = 77, p = 0.007, η2 = 0.05  n = 174, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Standard          Circle  N = 273, p = 0.000, η2 = 0.05  n = 83, p = 0.000, η2 = 0.09  n = 173, p = 0.000, η2 = 0.03  n = 17, p = 1, η2 = 0.00  Adjust  N = 263, p = 0.017, η2 = 0.01  n = 75, p = 0.029, η2 = 0.04  n = 171, p = 0.112, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Publish  N = 269, p = 0.105, η2 = 0.01  n = 81, p = 0.096, η2 = 0.03  n = 171, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Flemma  All participants  Beginner group  Intermediate group  Advanced group  Use  N = 277, p = 0.006, η2 = 0.01  n = 84, p = 0.137, η2 = 0.02  n = 176, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Move  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.014, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Collect  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.013, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Center  N = 272, p = 0.000, η2 = 0.09  n = 82, p = 0.000, η2 = 0.1  n = 173, p = 0.000, η2 = 0.09  n = 17, p = 1, η2 = 0.00  Teach  N = 279, p = 0.019, η2 = 0.01  n = 85, p = 0.084, η2 = 0.02  n = 177, p = 0.194, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Accept  N = 255, p = 0.008, η2 = 0.02  n = 70, p = 0.300, η2 = 0.02  n = 168, p = 0.034, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Maintain  N = 268, p = 0.003, η2 = 0.02  n = 74, p = 0.004, η2 = 0.06  n = 177, p = 0.340, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Develop  N = 268, p = 0.000, η2 = 0.03  n = 77, p = 0.007, η2 = 0.05  n = 174, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Standard          Circle  N = 273, p = 0.000, η2 = 0.05  n = 83, p = 0.000, η2 = 0.09  n = 173, p = 0.000, η2 = 0.03  n = 17, p = 1, η2 = 0.00  Adjust  N = 263, p = 0.017, η2 = 0.01  n = 75, p = 0.029, η2 = 0.04  n = 171, p = 0.112, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Publish  N = 269, p = 0.105, η2 = 0.01  n = 81, p = 0.096, η2 = 0.03  n = 171, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Among the beginner-group and intermediate-group participants, significant differences (p < 0.001) only existed among the number of correct responses to the base words center and circle, and correct responses to their associated inflectional forms (Table 3). Nonsignificant differences are present for nine (use, move, collect, teach, accept, maintain, develop, adjust, and publish) of the 11 flemmas. Furthermore, the effect sizes of the significant differences are small (η2 = 0.03–η2 = 0.1). Among the advanced-group participants (n = 17), no significant differences existed among the number of correct responses to the base words and correct responses to associated inflectional forms (Table 3). The general absence of statistically significant discrepancies among the participants’ ability to comprehend the base words and associated inflectional forms, and small effect sizes in the cases when statistically significant discrepancies were present indicate that the participants possess the ability to comprehend inflectional forms. Thus, the flemma is an appropriate general word counting unit for Japanese EFL learners. Research question 3 asks, if the flemma were adopted, what inflectional and derivational forms would the flemma overestimate or underestimate participants’ ability to comprehend? Research question 3 is addressed through investigating the difference between the accuracy of responses to base forms and individual associated inflectional and derivational forms using McNemar analysis. Considering the high threshold of 98 per cent knowledge of tokens within a text necessary for reading comprehension, the inclusion of an inflectional or derivational form that is significantly more difficult to comprehend than the base form is defined as overestimating the participants’ ability to comprehend a given form. The exclusion of a derivational form which is not significantly more difficult to comprehend than the base form is considered as underestimating learners’ ability to comprehend a derivational form. Table 4 indicates that, among all 279 participants, the adoption of the flemma would only underestimate the participants’ ability to comprehend the derivational forms user, teacher, and publisher, but not developer (φ 0.16) and adjuster (φ 0.1). The same results were found among beginner-group and intermediate-group participants. Table 4: The significance and effect size of differences in the number of participants who comprehend base forms and the number of participants who comprehend associated inflectional forms and derivational forms Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle (verb)  Adjust  Publish  -ed  0.01  0.0  0.0  0.08*  0.01  0.01  0.01  0.03    0.04*  0.01  0.01  -ing  0.0  0.01  0.02  0.12*  0.01  0.02  0.03  0.03    0.07*  0.02  0.01  -ing adjective                0.09*          have-ed  0.03  0.02  0.03  0.13*  0.03  0.03  0.03  0.03    0.07*  0.01  0.01  -er  0.02        0.01      0.16*      0.1*  0.01  -able  0.2*    0.2*                0.26*    -less  0.41*                        un-able                0.44*        0.41*  -erly          0.49*                -ingly    0.5*                      un-ing            0.59*              un-ed              0.67*      0.58*      non-                  0.74*        -ment    0.16*            0.23*      0.26*    -ful  0.28*                        -ized                  0.56*        -mentally                0.71*          -ability          0.78*  0.79*  0.76*          0.72*  -ally        0.21*                  -ance            0.38*  0.31*            mis-ed          0.41*              0.58*  anti-ment                0.7*          -alized        0.73*                  -age  0.77*                        counter-    0.84*                      sub-                  0.89*        mal-ed                      0.93*    semi-                    0.93*      en-ment                    0.94*      re-  0.12*          0.25*    0.16*    0.25*  0.23*    -tion      0.16*                    re-able  0.27*                        re-ing                        0.39*  -ic        0.58*                  pre-ed          0.53*                -ist        0.42*                  -ization                  0.59*        -ivization      0.79*                    Circle (noun)                    0.0      Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle (verb)  Adjust  Publish  -ed  0.01  0.0  0.0  0.08*  0.01  0.01  0.01  0.03    0.04*  0.01  0.01  -ing  0.0  0.01  0.02  0.12*  0.01  0.02  0.03  0.03    0.07*  0.02  0.01  -ing adjective                0.09*          have-ed  0.03  0.02  0.03  0.13*  0.03  0.03  0.03  0.03    0.07*  0.01  0.01  -er  0.02        0.01      0.16*      0.1*  0.01  -able  0.2*    0.2*                0.26*    -less  0.41*                        un-able                0.44*        0.41*  -erly          0.49*                -ingly    0.5*                      un-ing            0.59*              un-ed              0.67*      0.58*      non-                  0.74*        -ment    0.16*            0.23*      0.26*    -ful  0.28*                        -ized                  0.56*        -mentally                0.71*          -ability          0.78*  0.79*  0.76*          0.72*  -ally        0.21*                  -ance            0.38*  0.31*            mis-ed          0.41*              0.58*  anti-ment                0.7*          -alized        0.73*                  -age  0.77*                        counter-    0.84*                      sub-                  0.89*        mal-ed                      0.93*    semi-                    0.93*      en-ment                    0.94*      re-  0.12*          0.25*    0.16*    0.25*  0.23*    -tion      0.16*                    re-able  0.27*                        re-ing                        0.39*  -ic        0.58*                  pre-ed          0.53*                -ist        0.42*                  -ization                  0.59*        -ivization      0.79*                    Circle (noun)                    0.0      Note. Effect sizes (φ) of differences between the number of participants who comprehend base words and associated inflected or derived forms. * significant difference. The alpha values for comparisons were established using the Bonferroni adjustment. Table 4: The significance and effect size of differences in the number of participants who comprehend base forms and the number of participants who comprehend associated inflectional forms and derivational forms Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle (verb)  Adjust  Publish  -ed  0.01  0.0  0.0  0.08*  0.01  0.01  0.01  0.03    0.04*  0.01  0.01  -ing  0.0  0.01  0.02  0.12*  0.01  0.02  0.03  0.03    0.07*  0.02  0.01  -ing adjective                0.09*          have-ed  0.03  0.02  0.03  0.13*  0.03  0.03  0.03  0.03    0.07*  0.01  0.01  -er  0.02        0.01      0.16*      0.1*  0.01  -able  0.2*    0.2*                0.26*    -less  0.41*                        un-able                0.44*        0.41*  -erly          0.49*                -ingly    0.5*                      un-ing            0.59*              un-ed              0.67*      0.58*      non-                  0.74*        -ment    0.16*            0.23*      0.26*    -ful  0.28*                        -ized                  0.56*        -mentally                0.71*          -ability          0.78*  0.79*  0.76*          0.72*  -ally        0.21*                  -ance            0.38*  0.31*            mis-ed          0.41*              0.58*  anti-ment                0.7*          -alized        0.73*                  -age  0.77*                        counter-    0.84*                      sub-                  0.89*        mal-ed                      0.93*    semi-                    0.93*      en-ment                    0.94*      re-  0.12*          0.25*    0.16*    0.25*  0.23*    -tion      0.16*                    re-able  0.27*                        re-ing                        0.39*  -ic        0.58*                  pre-ed          0.53*                -ist        0.42*                  -ization                  0.59*        -ivization      0.79*                    Circle (noun)                    0.0      Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle (verb)  Adjust  Publish  -ed  0.01  0.0  0.0  0.08*  0.01  0.01  0.01  0.03    0.04*  0.01  0.01  -ing  0.0  0.01  0.02  0.12*  0.01  0.02  0.03  0.03    0.07*  0.02  0.01  -ing adjective                0.09*          have-ed  0.03  0.02  0.03  0.13*  0.03  0.03  0.03  0.03    0.07*  0.01  0.01  -er  0.02        0.01      0.16*      0.1*  0.01  -able  0.2*    0.2*                0.26*    -less  0.41*                        un-able                0.44*        0.41*  -erly          0.49*                -ingly    0.5*                      un-ing            0.59*              un-ed              0.67*      0.58*      non-                  0.74*        -ment    0.16*            0.23*      0.26*    -ful  0.28*                        -ized                  0.56*        -mentally                0.71*          -ability          0.78*  0.79*  0.76*          0.72*  -ally        0.21*                  -ance            0.38*  0.31*            mis-ed          0.41*              0.58*  anti-ment                0.7*          -alized        0.73*                  -age  0.77*                        counter-    0.84*                      sub-                  0.89*        mal-ed                      0.93*    semi-                    0.93*      en-ment                    0.94*      re-  0.12*          0.25*    0.16*    0.25*  0.23*    -tion      0.16*                    re-able  0.27*                        re-ing                        0.39*  -ic        0.58*                  pre-ed          0.53*                -ist        0.42*                  -ization                  0.59*        -ivization      0.79*                    Circle (noun)                    0.0      Note. Effect sizes (φ) of differences between the number of participants who comprehend base words and associated inflected or derived forms. * significant difference. The alpha values for comparisons were established using the Bonferroni adjustment. In contrast, the adoption of the flemma would overestimate the participants’ ability to comprehend inflectional forms when they are of a different part of speech to the part of speech of the most frequently used base form. As explained in the scoring section, for reasons of ecological validity, markers were instructed not to distinguish between parts of speech when marking the base form. Thus, for the base form circle, if participants produced either the Japanese verb or noun form for circle, the item was marked as correct. Significant differences were found between the base form center, and centered (φ, 0.08), centering (φ 0.12), and have centered (φ. 13); between the base form circle, and circled (φ 0.04), circling (φ 0.07), have circled (φ 0.07); and between develop and developing (the adjectival form) (φ 0.09). This same pattern with almost identical effect sizes was repeated among the beginner-group and intermediate-group participants. Despite the significant difference, the negligible (<0.1) or small (≥0.1) effect sizes indicate that the difference between the adoption of either the lemma or the flemma is minimal. Among the advanced-group participants, the adoption of the flemma does not overestimate the ability of any participant's to comprehend inflectional forms. The small number (n = 17) of advanced-group participants limits the possibility of a significant difference being found between participants’ ability to comprehend base forms and associated derivational forms. Thus, when investigating this issue, the criterion for a significant difference is a small effect size (φ = >0.1), or an accuracy rate of < 88 per cent. Of the 51 derivational forms, 32 derivational forms were found to be significantly more difficult for advanced-group participants to comprehend than the base form. With three exceptions, all of these derivational forms contain multiple affixes, or include affixes described as regular but infrequent affixes (Level 5 derivational forms) or frequent but irregular affixes (Level 6 derivational forms) by Bauer and Nation (1993). This indicates that if the flemma is adopted as a general word counting unit, it would underestimate the advanced-group participants’ ability to comprehend 19 of the 52 tested derivational forms. DISCUSSION Research question 1 asked if there is a significant difference in Japanese EFL learners’ ability to comprehend the base form, inflectional forms, and derivational forms, while research question 2 asked the same question, but only in relation to inflectional forms. Tables 1–4 indicate that, despite demonstrating comprehension of the base word, the participants’ ability to comprehend associated derivational forms was very much incomplete, and was even the case among advanced-group participants. Research question 3 investigated the degree of overestimation and underestimation of the participants' ability to comprehend inflectional forms and derivational forms which would result from the adoption of the flemma. It was found that only slight overestimation would result from the adoption of the flemma among all 279 participants, and beginner-group and medium-group participants. Furthermore, even among advanced-group participants, the adoption of the flemma would only underestimate their ability to comprehend 19 of the 52 tested derivational forms. Thus, this study suggests that WF6 is not an appropriate general written receptive word counting unit for Japanese EFL learners when reading, and that the flemma is a more appropriate general written receptive word counting unit for Japanese EFL learners. Thus, WF6 lexical written receptive knowledge estimates yielded from publically available tests, such as the VST and the NVLT, are overestimations when the participants in the present study are representative of other L1 EFL populations. This is because WF6 based vocabulary tests present examinees with the base form or most common form of the target word and not less commonly known derivational forms, and then if the target form is answered correctly, knowledge of the target form and all associated WF6 members is assumed. However, even if multiple-choice tests presented examinees with derivational forms, instead of base forms, overestimation might persist. This is because within the multiple-choice question stem examinees could potentially find the base form of a word within its derived form without knowing the meaning of the derived form, and then match the base word meaning with the correct multiple-choice answer. For example, in the following modified first 1,000-WF6 VST item, standard is replaced with standardization. An examinee who knows the meaning of standard and normal, but not the meaning of standardization, can find the base word standard within standardization, and then can correctly answer the multiple-choice item below. The test administrator might then wrongly assume that the examinee knows the derived form standardization and other derived -ize and -ization forms. Standardization: Standardization is sometimes a problem. A part of a shoe Marks at school Money people ask for The setting of normal levelsImplications for the accurate measurement of vocabulary knowledge. The accurate measurement of vocabulary size and levels can be addressed through three methods. First, researchers may try to establish what inflectional and derivational forms their participants know before creating appropriate word lists and vocabulary tests. To the best of my knowledge, no researchers have done this, and this is impractical because (i) the ability to comprehend inflectional and derivational forms was found to be base word dependent, (ii) extrapolating knowledge of inflectional and derivational forms containing only one affix to words containing multiple affixes is problematic (17 participants who correctly produced the meaning of both reuse and usable could not produce the meaning of reusable), and (iii) the use of different counting units within a single study inhibits the comparison of results among participants and between studies. Nation and Coxhead (2014) state that prior to Bauer and Nation (1993), it was clear that a standard word family unit was needed, to enable consistency between different word families, and the failure to do so was a contributing factor which renders most twentieth-century research on vocabulary size and levels virtually useless or grossly misleading. A second possible solution is the use of a word counting unit including derivational forms known by most Japanese EFL learners. This study found that only some, not all, derivational -er forms were not significantly more difficult to comprehend than the associated base form. Thus, simply adopting the flemma would seem pragmatic. A third solution is to adopt a smaller counting unit, possibly the flemma for Japanese EFL learners. The adoption of the flemma has various other advantages. First, the flemma is an easily understood unit allowing readers to easily comprehend and evaluate research findings, thus, making the replication and comparison of studies more feasible (Schmitt 2010). In contrast, WF6 is more problematic than other counting units because it is unclear which forms belong in the WF6 unit. What about infrequent and hyphenated forms such as teacher-like or developmentwise? Second, to facilitate the comparison of receptive and productive studies, the flemma is a reasonable compromise. Nation (2006b), Schmitt (2010), Nation and Webb (2011), and Nation (2013) have argued that the WF6 unit should not be used when measuring productive vocabulary knowledge. Implications for research The validity of the research paradigm that short corpora-derived lists of high-frequency WF6 lists should be deliberately taught and learned because they cover such a large percentage of the tokens that English language learners will actually encounter is questioned by this study. Similarly, this study questions the validity of studies which cite Hu and Nation (2000) when estimating the WF6 vocabulary level necessary to comprehend texts. This is because these studies are based on the coverage provided by WF6 and not the flemma, and thus assume that learners are able to comprehend all WF6 forms. Thus, the results of this study argue that inferences made in studies based on the coverage of WF6 are of limited validity unless evidence exists that participants and learners are able to comprehend WF6 inflectional and derivational forms. This argument is supported by the figures in Brown (2017) which show that text from the BNC which has 98 per cent coverage on the basis of WF6 has only 85 per cent coverage on the basis of the flemma, which this study indicates is an appropriate word counting unit for Japanese EFL learners. LIMITATIONS The above results should be viewed in the context of the limitations of this study. Although the participants in this study were from a wide range of proficiencies, they are all L1 Japanese EFL learners. Thus, generalizing the results of this study to learners of different proficiency levels or L1s, particularly Latin- or Germanic-based languages is problematic. Second, it would have been preferable to test a larger range of inflectional and derivational forms. Third, the participants’ ability to comprehend derivational and inflectional forms of only 12 words was investigated. This is an issue because this study found that the degree to which participants comprehend inflectional forms and derivational forms consisting of the same affix was base word dependent. Fourth, while this study refers to the decreases in the coverage provided by the flemma relative to the word family in the BNC, the degree to which WF6 and the flemma provide varying coverage of a range of individual texts of varying difficulties and types is unknown, and it is hoped that future research investigates this. Finally, the findings of this study are limited to the comprehension of orthographic forms when reading, and so further research into an appropriate counting unit when listening, when speaking, and when writing is necessary. CONCLUSION This article adds to the existing L2 literature on EFL learners’ knowledge of inflectional and derivational forms by directly addressing the question of the appropriateness of WF6 and the flemma as a written receptive word counting unit with Japanese EFL learners. Results indicate that despite comprehending base words, the participants’ ability to comprehend associated derivational forms was very limited. Thus, the flemma was found to be more appropriate than WF6 as a word counting unit for these participants, including those who demonstrated mastery of the base forms of the fourth or fifth 1,000 BNC/COCA WF6 bands. Even among the advanced-group participants, considering the 98 per cent threshold for reading comprehension, it is not possible to recommend the adoption of WF6. Finally, results indicate that participants’ ability to comprehend inflectional and derivational forms was base word dependent. Thus, when diagnosing learner’s knowledge of affixes, inferences should be made with caution. These findings, combined with those of Schmitt and Meara (1997), Mochizuki and Aizawa (2000), Sasao and Webb (2015), and Ward and Chuenjundaeng (2009) question the validity of the research and vocabulary tests that have adopted WF6, the use of WF6 word lists, and the research paradigm that short corpora-derived lists of high-frequency words should be deliberately taught and learned because they cover such a large percentage of the tokens that English language learners will encounter. SUPPLEMENTARY DATA Supplementary material is available at Applied Linguistics online. Acknowledgements The author would like to thank the editor and reviewers whose comments and suggestions have greatly improved this article. The author would also like to thank Paul Nation, Batia Laufer, Kiyomi Yoshizawa, Sawako Hamatani, Brandon Kramer, Geoffrey Pinchbeck, Laura Huston, Steve Porritt, Tim Stoeckel, and Dale Brown for their feedback on this article. The author has special thanks for David Beglar who greatly improved the quality of the article. Any remaining faults are the sole responsibility of the author. Funding This work was supported by JSPS KAKENHI (grant number JP 16K16890). Conflict of interest statement. None declared. NOTES ON CONTRIBUTOR Stuart McLean is an instructor at Osaka Jogakuin University. He holds an M.S.Ed. (TESOL), a Ph.D. in Forensic Medicine, and a P.G.C.E, and is an Applied Linguistics doctoral student at Kansai University. He has published in Reading in a Foreign Language, Vocabulary Learning and Instruction, Language Teaching Research, TESOL Quarterly, Applied Linguistics, and Language Assessment Quarterly. Address for correspondence: Osaka Jogakuin University, 2-26-54 Tamatsukuri, Osaka-shi, Osaka, Japan 540-004. <stuart93@me.com> Notes 1 While this study refers to the decreases in the coverage provided by the flemma relative to the word family in the BNC, the degree to which WF6 and the flemma provide varying coverage of a range of individual texts of varying difficulties and types requires investigation. 2 Prior to participants completing the inflectional and derivational forms comprehension test, participants completed a productive test (L1 meaning to L2 form translation test) for the same target items. Participants were not handed the comprehension test until the productive test was completed and returned to the researcher. The sequence of measuring productive and then receptive knowledge of inflectional and derivational forms, the large number of items in each test (K = 100), and the test items being ordered differently in each test ensure that test effect, if present, is minimal (Nation, 2013). Interviews with the participants indicated that no test effect was present. This is an example of a productive test item: 使用可能な(useable): The computer is now ________________ . The translation in parentheses was not visible to examinees. 3 In the 12 target WF6 sets, the standard WF6 set has no inflectional forms. Thus, the number of WF6 sets among which a difference in participants’ ability to comprehend the base forms, and inflectional forms was 11 and not 12. REFERENCES Bauer L. Nation P.. 1993. ‘ Word families,’ International Journal of Lexicography  6: 253– 79. Google Scholar CrossRef Search ADS   Brown D. (2017). Examining the word family through word lists (in preparation). Brown J. D. Coombe C.. 2015. Research in language Teaching and Learning . Cambridge University Press. Francis W. Kucˇera H.. 1982. Frequency analysis of English usage . Houghton Mifflin. Gardner D. 2007. ‘ Validating the construct of word in applied corpus-based vocabulary research—A critical survey,’ Applied Linguistics  28: 241– 65. Google Scholar CrossRef Search ADS   Green B. Salkind J.. 2013. Using SPSS for Windows and Macintosh . Prentice Hall Press. Gyllstad H. Vilkaite L. Schmitt N.. 2015. ‘ Assessing vocabulary size through multiple-choice formats: Issues with guessing and sampling rates,’ ITL International Journal of Applied Linguistics  166: 278– 306. Google Scholar CrossRef Search ADS   Hu M. Nation I. S. P.. 2000. ‘ Vocabulary density and reading comprehension,’ Reading in a Foreign Language  13: 403– 30. Laufer B. 1997. ‘The lexical plight in second language reading: Words you don't know, words you think you know, and words you can't guess’ in Coady J. Huckin T. (eds): Second Language Vocabulary Acquisition: A Rationale for Pedagogy , pp. 20– 34. Cambridge University Press. Laufer B. Goldstein Z.. 2004. ‘ Testing vocabulary knowledge—Size, strength, and computer adaptiveness,’ Language Learning  54: 399– 436. Google Scholar CrossRef Search ADS   Macalister J. Webb S.. 2013. ‘ A response,’ TESOL Quarterly  47: 852– 5. Google Scholar CrossRef Search ADS   Meara P. Buxton B.. 1987. ‘ An alternative to multiple choice vocabulary tests,’ Language Testing  4: 142– 54. Google Scholar CrossRef Search ADS   McLean S. Kramer B.. 2015. ‘ The creation of a new vocabulary levels test,’ Shiken  19: 1– 11. McLean S. Ishii T. Stoeckel T. Bennett P. Matsumoto Y.. 2016. ‘ An edited version of the first eight 1,000-word frequency bands of the Japanese-English version of the vocabulary size test,’ The Language Teacher  40: 3– 7. Mochizuki M. Aizawa K.. 2000. ‘ An affix acquisition order for EFL learners—An exploratory study,’ System  28: 291– 304. Google Scholar CrossRef Search ADS   Nagy W. Diakidoy I.-A. Anderson R.. 1993. ‘ The acquisition of morphology: Learning the contribution of suffixes to the meanings of derivatives,’ Journal of Reading Behavior  25: 155– 80. Google Scholar CrossRef Search ADS   Nation I. S. P. 2001. Learning Vocabulary in Another Language . Cambridge University Press. Google Scholar CrossRef Search ADS   Nation I. S. P. 2006a. BNC-based Word Lists . Victoria University of Wellington. Nation I. S. P. 2006b. ‘ How large a vocabulary is needed for reading and listening?,’ The Canadian Modern Language Review  6: 59– 82. Google Scholar CrossRef Search ADS   Nation I. S. P. 2012. The Vocabulary Size Test. Nation I. S. P. 2013. Learning Vocabulary in Another Language . Cambridge University Press. Google Scholar CrossRef Search ADS   Nation I. S. P. 2014. ‘ How much input do you need to learn the most frequent 9,000 words?,’ Reading in a Foreign Language  26: 1– 16. Nation I. S. P. Beglar D.. 2007. ‘ A vocabulary size test,’ The Language Teacher  31: 9– 13. Nation I. S. P. Coxhead A.. 2014. ‘ Vocabulary size research at Victoria University of Wellington, New Zealand,’ Language Teaching  47: 398– 403. Google Scholar CrossRef Search ADS   Nation I. S. P. Webb S.. 2011. Researching and Analyzing Vocabulary . Heinle. Pinchbeck G. G. 2014. Lexical Frequencies Profiling of Canadian High School Diploma Exam Expository Writing: L1 and L2 Academic English. Roundtable presentation at American Association of Applied Linguistics, Toronto, Ontario. Rovai A. P. Baker J. D. Ponton M. K.. 2013. Social Science Research Design and Statistics: A Practitioner's Guide to Research Methods and IBM SPSS . Watertree Press LLC. Sasao Y. Webb S.. 2015. ‘ The Word Part Levels Test,’ Language Teaching Research  19: 667– 86. Google Scholar CrossRef Search ADS   Schmitt N. 2010. Researching Vocabulary: A Vocabulary Research Manual . Palgrave Press. Google Scholar CrossRef Search ADS   Schmitt N. Cobb T. Horst M. Schmitt D.. 2015. ‘ How much vocabulary is needed to use English? Replication of Van Zeeland & Schmitt (2012), Nation, (2006), and Cobb (2007),’ Language Teaching  1– 15. Schmitt N. Jiang X. Grabe W.. 2011. ‘ The percentage of words known in a text and reading comprehension,’ The Modern Language Journal  95: 26– 43. Google Scholar CrossRef Search ADS   Schmitt N. Schmitt D. Clapham C.. 2001. ‘ Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test,’ Language Testing  18: 55– 88. Google Scholar CrossRef Search ADS   Schmitt N. Meara P.. 1997. ‘ Researching vocabulary through a word knowledge framework,’ Studies in Second Language Acquisition  19: 17– 36. Google Scholar CrossRef Search ADS   Sinclair J. 2004. How to Use Corpora in Language Teaching . John Benjamins Publishing Company. Google Scholar CrossRef Search ADS   Swenson E. West M.. 1934. On the Counting of New Words in Textbooks for Teaching Foreign Languages No. 1 . University of Toronto. Tyler A. Nagy W.. 1989. ‘ The acquisition of English derivational morphology,’ Journal of Memory and Language  28: 649– 67. Google Scholar CrossRef Search ADS   Ward J. Chuenjundaeng J.. 2009. ‘ Suffix knowledge: Acquisition and applications,’ System  37: 461– 9. Google Scholar CrossRef Search ADS   Webb S. 2010. ‘ A corpus driven study of the potential for vocabulary learning through watching movies,’ International Journal Of Corpus Linguistics  15: 497– 519. Google Scholar CrossRef Search ADS   Webb S. A. Sasao Y.. 2013. ‘ New Directions In vocabulary testing,’ RELC Journal  44: 263– 77. Google Scholar CrossRef Search ADS   © Oxford University Press 2017 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Applied Linguistics Oxford University Press

Evidence for the Adoption of the Flemma as an Appropriate Word Counting Unit

Applied Linguistics , Volume Advance Article – Feb 4, 2017

Loading next page...
 
/lp/ou_press/evidence-for-the-adoption-of-the-flemma-as-an-appropriate-word-59ySYOAnoK
Publisher
Oxford University Press
ISSN
0142-6001
eISSN
1477-450X
D.O.I.
10.1093/applin/amw050
Publisher site
See Article on Publisher Site

Abstract

Abstract An important gap in the field of second language vocabulary research concerns the ability of Asian learners of English as a Foreign Language (EFL) to comprehend inflectional and derivational word family members. Japanese EFL learners (N = 279) were divided into three lexical proficiency groups, and their ability to comprehend inflectional and derivational English forms was measured with an English to Japanese translation test. A significant difference among the participants' ability to comprehend 12 base forms, associated inflected forms, and associated derived forms was found across the three proficiency groups, and even among participants who demonstrated mastery of the first 4,000 or 5,000 base forms of English. The flemma, a word's base form and associated inflectional forms, was found to be an appropriate word counting unit for most participants. Results are important because corpus research findings demonstrate that in cases where the word family provides 98 per cent coverage of texts, the flemma only provides 85 per cent coverage of the same texts. Thus, considering the detrimental impact to reading comprehension from only small decreases in the percentage of known tokens within a text, the results question the inferences made in word family-based research. INTRODUCTION An important gap in the field of second language (L2) vocabulary research concerns English as a Foreign Language (EFL) learners’ ability to comprehend inflectional and derivational forms. This is of importance as the choice of what inflectional and derivational forms are included with a word’s base form has major ramifications for the validity of corpora-derived word lists, word list-based vocabulary tests, and research based upon these (Gardner 2007). The influence of corpora and corpus-based research on educational theories, assessment, and pedagogic practices in L2 settings is significant (Nation 2001; Sinclair 2004). Vocabulary levels research, including investigations into the coverage of text high-frequency words provide, is an example of how corpus-based vocabulary investigations have shaped pedagogy. Vocabulary levels research argues that short corpora-derived lists of high-frequency words should be deliberately taught and learned because they cover such a large percentage of the tokens that English language learners encounter. Further, it is argued that learners must be able to comprehend a relatively small number of high-frequency, high-coverage words for incidental vocabulary growth; otherwise, achieving unassisted comprehension of texts and subsequently contextual word learning would be inhibited (Laufer 1997; Nation 2001). The validity of this research paradigm and corpus-based vocabulary applications is dependent on researchers appropriately operationalizing the word unit (Gardner 2007). Further, when corpus-based vocabulary findings are used to inform or support language acquisition, there is the additional concern of whether researcher-based conceptualizations of a word, the criteria used to group and count words, matches the psychological realities of a word, the knowledge of words possessed by the target language users. Thus, learners’ ability to comprehend inflectional and derivational forms requires direct investigation. In L2 lexical and reading research, the most commonly adopted word counting unit is the word family, which consists of the base word and inflected and derived forms from Levels 2–6 of Bauer and Nation’s (1993) affix criteria (hereafter referred to as WF6). Bauer and Nation’s criteria and first language (L1) studies (Tyler and Nagy 1989; Nagy et al. 1993) have been used to justify the use of WF6 as a general word counting unit in both L1 and L2 settings for word list creation (Nation 2006a), vocabulary size tests (i.e., Nation and Beglar 2007), vocabulary levels tests (i.e., McLean and Kramer 2015), and lexical research (Nation 2006b; 2014). Bauer and Nation’s criteria by which English inflectional and derivational affixes were arranged into a graded set of seven levels were created to guide teaching and learning, and standardize vocabulary load and size research. Level ordering divides affixes into a number of types based on their phonological and morphological behavior to determine the level at which a particular affix should be placed. Frequency: The frequency with which an affix occurs. Productivity: The probability that the affix will be used to form new words. Predictability: The degree of predictability of the meaning of the affix. Written base form regularity: The predictability of changes in the written form of the base when the affix is added. Spoken base form regularity: The amount of change in the spoken form of the base when the affix is added. Affix spelling regularity: The predictability of written forms of the affix. Affix form regularity: The predictability of spoken forms of the affix. Regularity of function: The degree to which the affix attaches to a base of known form-class and produces a word of known form-class. Bauer and Nation’s (1993) explanations of a word family for receptive purposes and the operationalization of the word family in research have led to the word family not consisting of a number of possible levels, but commonly being operationalized as WF6 in studies of receptive language use. Bauer and Nation first defined a word family as ‘a base word and all its derived and inflected forms that can be understood by a learner without having to learn each form separately’ (p. 253). Later, Bauer and Nation (1993) stated that ‘watch, watches, watched, and watching may all be members of the same word family for a learner with a command of the inflectional suffixes of English. As a learner's knowledge of affixation develops, the size of the word family increases’ (p. 253). This explanation narrows the word family to a single unit that differs over time and between learners, and thus inhibits the comparison of learners within a single study, and the comparison of research findings between studies. Contrary to this temporal view of the word family that limits the definition of the word family to only forms which are known by a learner is Bauer and Nation’s statement that, ‘The important principle behind the idea of a word family is that once the base word or even a derived word is known, the recognition of other members of the family requires little or no extra effort’ (p. 253). Bauer and Nation provided an example of this principle and stated that once the meaning of marmelize is explained, readers ‘will easily understand He fully deserved his marmelization, which contains a new word-form marmelization’ (p. 253). While this might be true for some learners, less proficient learners might not understand that marmelization is a noun, and might not recognize the meaning of this new word-form. The British National Corpus (BNC) word list (Nation 2006a) and the British National Corpus/Corpus of Contemporary American English (BNC/COCA) word list (Nation 2012), which both utilize WF6, have contributed to the word family commonly being operationalized as WF6 in L2 lexical and reading research. While Nation (2006a) stated that WF6 is appropriate for advanced learners without defining advanced learners, Nation (2012: 3) simply stated that ‘The word family lists group items together that would be perceived as the same words for the receptive skills of listening and reading’. Nation (2014) described the use of WF6 as conservative, when describing the number of words learners with a vocabulary level as low as 1,000 WF6 need to know to comprehend materials and incidentally learn new words from context. A circular argument that WF6 is an appropriate general word counting unit because Bauer and Nation state so is present in the description of the increasingly used Vocabulary Size Test (VST) (Nation and Beglar 2007: 10): ‘all of the family members at Level 6 meet the criteria of regularity, frequency, productivity, and predictability used in developing the word family levels’. Macalister and Webb (2013: 853) stated that the WF6 unit was used in their study, as the study was ‘on receptive vocabulary (comprehension), and for this the word family is “the most sensible unit” (Nation and Webb 2011: 136), as it assumes that a reader who knows one or two members of the word family should be able to recognize and understand other members of the family’. In other cases, the use of WF6 is based on the view that knowing one member of a word family is likely to facilitate recognition of the other members, despite no empirical evidence supporting this. ‘It is important to note that using word families as the unit of counting means that there is the assumption that if the head word is known, all members of the word family will also be known. There is a strong argument that this is true for receptive knowledge of word families because knowing one member of a word family is likely to facilitate recognition of other members’ (Webb 2010: 507). A further issue with the adoption of WF6 in Japanese EFL settings is that evidence for its adoption comes from L1 research and not L2 research. Evidence that WF6 is a psychologically real unit comes from L1 investigations (Tyler and Nagy 1989; Nagy et al. 1993). However, these studies describe incomplete knowledge of derivational forms among high-school students in their L1. Fourth graders were able to recognize familiar L1 English stems in derivatives, and only eighth graders possessed knowledge of the syntactic properties of derivational suffixes (Tyler and Nagy 1989). Furthermore, Nagy et al. (1993) described significant development in suffix knowledge of L1 English learners continuing between the fourth grade and high school, and then even through high school among some learners. Thus, L1 studies present a mixed case for the use of WF6 in L2 English settings, unless we assume that most L2 English learners’ knowledge of derivational forms is greater than L1 English high school students. If the use of WF6 overestimates EFL learners’ knowledge of inflectional and derivational forms, the overestimation will be greatest among low-proficiency learners, as a significant positive correlation has been reported between vocabulary size and affix knowledge (Schmitt and Meara 1997; Mochizuki and Aizawa 2000). This is critical, as the highest frequency words possess the largest number of inflectional and derivational forms. For instance, Brown (2017) in an analysis of the first five 1,000-WF6 bands of the BNC lists shows that the number of word forms among the first, second, third, fourth, and fifth 1,000 words of the BNC lists are 6.348, 5.593, 4.517, 4.288, and 3.992, respectively. Further, Brown shows that texts with 95 per cent coverage on the basis of WF6 have only 82.3, 86.6, 89.5, and 91.2 per cent coverage when Levels 2-, 3-, 4-, and 5-word family forms are included, respectively.1 Similarly, texts with 98 per cent coverage on the basis of WF6 only have 84.9, 89.3, 92.4, and 94.1 per cent coverage if Levels 2-, 3-, 4-, and 5-word family forms are included, respectively. Hu and Nation (2000) and Schmitt, Jiang, and Grabe (2011) stated that such drops in coverage inhibit reading comprehension. An alternative to WF6 is the lemma or flemma. The lemma consists of a headword and its inflected, irregular, and reduced forms (n't) that are of the same part of speech (Francis and Kucˇera 1982). The English inflected forms are plural, third person singular present tense, past tense, past participle, -ing, comparative, superlative, and possessive forms. While the word family develop consists of develop, develops, developed, developing, developable, development, developments, developmental, undevelopable, developer, developers, undeveloped, developmental, developmentally, developmentwise, semideveloped, antidevelopment, redevelop, and predevelopment, the lemma develop only consists of develop, develops, developed, and developing. Swenson and West’s (1934) concept of learning burden underpins the lemma, positing that the learning burden or difficultly in inferring the meaning of inflected forms from the base form is minor. The flemma, unlike the lemma, groups identical forms of different parts of speech. Thus, the verb developed and the adjective developed are different lemmas, but members of the same flemma (Pinchbeck, 2014). Schmitt (2010) argued that the lemma might be the best general unit of counting for four reasons. First, what forms are included within a lemma is straightforward relative to the word family, facilitating the easy comprehension of research studies. Further, opinions differ on what to do with hyphenated forms and rare forms such as -wise or -like. Second, the comparatively straightforward lemma makes the replication and comparison of studies more feasible. Third, the lemma is a pragmatic compromise when counting both receptive and productive vocabulary use, facilitating the comparison of receptive and productive studies. No researchers have suggested the adoption of WF6 when measuring productive lexical knowledge. Finally, Schmitt argued that, relative to WF6, the lemma provides more realistic estimates of the large number of words necessary to function in English. This notion is supported by existing L2 research (Schmitt and Meara 1997; Mochizuki and Aizawa 2000; Ward and Chuenjundaeng 2009; Sasao and Webb 2015) which indicates that Thai and Japanese EFL learners comprehend a limited number of WF6 derivational forms. Evidence that the flemma is a more appropriate general written receptive word counting unit than WF6 with Japanese EFL learners comes from four studies. Schmitt and Meara (1997) reported that Japanese university and high school participants demonstrated meaning-recognition knowledge of 52.85 per cent of affixes at Levels 2–6 of Bauer and Nation’s (1993) affix criteria, including -ed (92 per cent), -able (30 per cent), -ence (58 per cent), and -al (10 per cent). Mochizuki and Aizawa (2000), using pseudoword base forms and multiple-choice tests, found that Japanese university and high school participants knew (meaning recognition) on average 7.24 (56 per cent) of 13 prefixes, and 10.70 (67 per cent) of 16 suffixes. Ward and Chuenjundaeng (2009) investigated 257 Thai learners’ ability to recall the meaning of both the base and derived forms of 32 target words. Among intermediate learners (n = 167) only 41.8, 33.0, 40.6, and 59.5 per cent demonstrated knowledge (meaning-recall) of the derived -tion, -er, -ment, and -ity forms, respectively. Among more proficient medical students (n = 90) only 69.2, 69.5, 56.0, and 42.8 per cent demonstrated knowledge (meaning-recall) of the derived -tion, -er, -ment, and -ity forms, respectively. Finally, Sasao and Webb (2015) investigated 1,348 participants’ meaning-recognition knowledge of 118 affixes. The results from multiple-choice tests revealed the relative difficulty of 118 affixes and indicated that the participants’ knowledge (form, meaning, and use) of the 118 affixes was far from complete. These four studies indicate that their participants had limited receptive knowledge of inflectional and derivational forms. However, the shared and/or individual limitations of these studies prevent valid conclusions being made regarding the degree to which Japanese learners of different proficiencies are able to comprehend inflectional and derivational forms. First, only Sasao and Webb (2015) stated the reliability of data collected. Second, Schmitt and Meara (1997) and Ward and Chuenjundaeng (2009) only tested knowledge of suffixes, and not prefixes. Third, none of the four studies tested knowledge of words with multiple affixes. For example, the word reusable consists of the base form, use, and the affixes re- and -able. Brown (2017) found that 26.8 per cent of the total number of word forms from the first 5,000 WF6 of the BNC contained two or more affixes. Fourth, the use of multiple-choice instruments (Schmitt and Meara 1997; Mochizuki and Aizawa 2000; Sasao and Webb 2015) overestimates learners’ knowledge and limits the ecological validity of inferences. Sasao and Webb stated that guessing with multiple-choice instruments overestimates learners’ knowledge and so corrected for lucky guessing (unexpected success by low-ability respondents). However, this procedure fails to distinguish between idiosyncratic vocabulary knowledge and guessing, and the accuracy of this correction is unknown. Thus, while Mochizuki and Aizawa, Sasao and Webb, and Schmitt and Meara made major contributions to our understanding of learners’ knowledge of affixes relative to other affixes through the use of multiple-choice tests, the degree to which the data collected in these studies accurately represent the learners’ actual ability to comprehend inflectional and derivational forms is unknown. Furthermore, the ecological validity of using a multiple-choice test (meaning recognition) to operationalize students ability to recall the meaning of derived forms as when reading has been questioned by Gyllstad, Vilkaite, and Schmitt (2015). This is because meaning recognition requires a lower threshold of lexical knowledge than meaning recall (Laufer and Goldstein 2004; Schmitt 2010; Nation 2013; Nation and Webb 2011), and when reading, the meanings of words are recalled, and not recognized from four possible options. Finally, Mochizuki and Aizawa (2000) and Sasao and Webb (2015) also investigated their participants’ metalinguistic knowledge of affixes (i.e., the part of speech of affixes) and not their ability to comprehend inflectional and derivational forms. The limitations of the existing research fuel continued calls for research that directly investigates the appropriateness of WF6: ‘we agree on the need for further research on the actual performance of second language learners when they encounter members of a word family’ (Macalister and Webb 2013: 853). Thus, this study investigates Japanese EFL learners’ ability to comprehend (produce the L1 meaning from the L2 form) inflectional and derivational WF6 forms consisting of a single affix or multiple affixes, and presents the following three research questions. RQ1 Is there a significant difference in Japanese EFL learners’ ability to comprehend the base form, inflectional forms, and derivational forms of the same word family? RQ2 Is there a significant difference in Japanese EFL learners’ ability to comprehend the base form and inflectional forms of the same word family? RQ3 What inflectional and derivational forms does the flemma overestimate or underestimate participants’ ability to comprehend? These research questions will be addressed by first investigating the participants as a single group, and then as three groups classified by lexical proficiency. METHODOLOGY Participants The participants (N = 279), aged between 18 and 24 years, were Japanese EFL university students who had studied English formally for six years in secondary school. In total, 185 participants (87 females and 98 males) were first-year university students enrolled in 3 hours per week of English language courses. They belonged to their institution’s intermediate English stream. In total, 50 participants (4 females and 46 males) were first-year university students enrolled in 3 hours per week of English language courses. They belonged to their institution’s advanced English stream. In total, 21 participants (16 females and 5 males) were second-year students who had elected to study in upper advanced English classes, and were enrolled in 3.0–4.5 hours of English language courses per week. In total, 23 participants, 10 third-year (9 females and 1 male) and 13 fourth-year (12 females and 1 male) students, were English majors. An additional 19 students failed to complete both instruments used in this study; thus, their data were not included. The participants came from intact classes taught by the researcher, and they gave written consent for the data in this research to be used. The majority of the participants had not completed a standardized English proficiency test. However, the participants’ New Vocabulary Levels Test (NVLT) scores (McLean and Kramer 2015) provide an indication of the participants’ written receptive meaning-recognition vocabulary knowledge by which they were separated into three groups. The beginner group (n = 85) consists of participants who failed to score 22 or more out of 24 on the second 1,000-word band of the NVLT. The intermediate group (n = 177) scored 22 or more on each of the second and third 1,000-word bands, but failed to score 22 or more on the fourth 1,000-word band. The advanced group (n = 17) scored 22 or more on the fourth and/or the fifth 1,000-word bands. While a cutoff score of 24 out of 24 would have been preferable for mastery, such a strict cutoff score would not allow for measurement error. A one-way analysis of variance (ANOVA) with groups as the independent variable and NVLT scores as the dependent variable found that the NVLT scores among these three groups differed significantly, F(2, 276) = 126.253, p = 0.000, with significant differences (p < 0.001) in NVLT scores between the advanced group and the intermediate group, the advanced group and the beginner group, and the intermediate group and the beginner group. Only the advanced-group participants had all completed a standardized test, and had shared learning experiences. They were all third-year or fourth-year English majors who had spent a year studying abroad in English-medium classes at universities in native English-speaking nations, and possessed English proficiencies very rarely met even at graduate school level in Japan, with a mean TOEIC score of 872. These participants had correctly answered at least 22 out of 24 multiple-choice vocabulary base word items from the 4,000 and/or 5,000 BNC/COCA WF6 bands. Webb and Sasao (2013) refer to the first 5,000 words of the BNC/COCA as the greatest range in vocabulary learning for the majority of L2 learners. Instruments Inflectional and derivational forms comprehension test This study sets out to measure the participants’ ability to comprehend the inflectional and derivational forms of known base words. Selecting target base forms too difficult for the participants would decrease the amount of data available for analysis. Thus, knowledge of 20 high-frequency words from the first 2,000 base words of the BNC by 17 low-proficiency participants was investigated. Twelve words (use, move, collect, center, teach, accept, maintain, develop, standard, circle, adjust, and publish) were known by all participants. The 12 words possess too many inflectional and derivational forms to be tested in their entirety. Thus, 87 inflectional and derivational forms were selected with the following criteria. Inflectional and derivational forms are from Levels 2–6 of Bauer and Nation’s (1993) criteria. Target forms include affixes that can be attached to free morphemes. However, some items consisted of multiple affixes that can be attached to free morphemes. For example, the form standardization consists of the affixes -ize and -ation. The derivational forms include affixes present in Nation’s (2001) list of derivational affixes for learners. Where possible, the same affix is present in target words from different word families to investigate possible variations in participants’ ability to comprehend different words containing the same affixes. For example, -ment was present in the items movement, adjustment, and development. The test did not include inflectional forms marking plurals (-s) and the third person (-s, -es) because Japanese does not mark these forms, making the testing of them in a translation test problematic, and because they have minimal impact on reading comprehension. The resulting 100-item comprehension (L2 form to L1 meaning translation) test (see Supplementary Materials) presented the target words in a context sentence using vocabulary from the first 2,000 WF6 of the BNC/COCA (Nation 2012), or other words (including loanwords) that piloting indicated that the participants would comprehend (panda, vocabulary, text, snack, network, and appropriate). Here is an example of one item: The computer is now usable = _____________. Instructions for the tests were presented in the participants’ L1, Japanese. Instructions accompanying the test were as follows.2Please translate the underlined word into Japanese. The answer can be expressed as a single word or multiple words. Please pay attention to the form of the underlined word. Please ensure that your writing is easy to read. The items belonging to a single WF6 were presented in sets. However, the order in which target WF6 members were presented differed to prevent the participants from guessing the meaning of the target form based on their order. For example, the past tense form did not always follow the base word (see Supplementary materials online). The test was completed within 30 minutes in class and was administered by the participants’ teacher and author of this article. Separating students into lexical proficiency groups The NVLT (McLean and Kramer 2015) consists of 24 multiple-choice items per 1,000-BNC/COCA WF6 band for the first five 1,000-word bands. The NVLT was administered by the author one week after the inflectional and derivational forms comprehension test, and the participants completed the NVLT within 30 minutes. Examinees select the Japanese response with the closest meaning to the target word from four options. A bilingual Japanese and not monolingual English version of the NVLT was given, as bilingual multiple-choice tests increase the unidimensionality of data collected and are completed more quickly (McLean et al. 2016). The test instructions were written in the examinees’ native language (Japanese) to support rapid and accurate comprehension. Examinees were instructed to skip questions they had absolutely no idea of. An example item is shown below. The translations in parentheses were not visible to examinees. School: This is a big school. 1 銀行 (bank) 海の動物 (sea animal) 学校 (school) 家 (house) Scoring The NVLT and the inflectional and derivational forms comprehension test answers were marked dichotomously. The NVLT data yielded a KR20 reliability estimate of 0.88. One Japanese native speaker, a teacher of English, marked all of the inflectional and derivational forms comprehension tests. Two further native speakers of Japanese, teachers of English, marked a randomly selected subset of 20 per cent of the inflectional and derivational forms comprehension tests. Using Kappa analysis, the interrater reliability values established among the three markers were all ≥0.91. The KR20 reliability estimate of the entire inflectional and derivational comprehension test data set marked by the main marker was 0.93, with estimates of 0.97 and 0.97 from data marked by the second and third marker, respectively. If the L1 Japanese meaning produced by the participants demonstrated accurate knowledge of the target word including the meaning of the target form’s affix(es), markers marked the answer as correct. Importantly, when marking the translations of the base forms, the answer was marked as correct if a participant answered with the noun form rather than the intended verb form for three reasons. First, the majority of vocabulary size tests and vocabulary levels tests are multiple-choice or Yes/No tests. The Yes/No tests (Meara and Buxton 1987) and Vocabulary Levels Test (VLT) (Schmitt et al. 2001) present target words in a decontextualized form. Thus, examinees who do not know that center can be a verb, but know the meaning of the noun center will state that they know the word center on a Yes/No test, or they can correctly select a multiple-choice answer on the VLT. Similarly, multiple-choice tests such as the VST and NVLT commonly present items in a decontextualized form and then in a sentence (e.g. see: They saw it). Third, and most importantly, word family and flemma-based lists group different parts of speech. Thus, vocabulary lists, tests, lexical profiling tools, and research based on them do not distinguish between parts of speech when estimating lexical knowledge. As a result, for the purpose of ecological validity, these answers were marked as correct when participants answered with a noun form instead of a verb form. Data analysis The following analysis is of data collected from only inflectional form items and derivational form items that the participants first correctly translated the associated L2 English base form into the L1 Japanese meaning. Thus, if a participant failed to comprehend the meaning of publish, that participant’s responses to the base form, inflectional and derivational members of the publish WF6 were excluded from the analysis. If the same participant correctly produced the L2 meaning of use, that participant’s responses to use WF6 members were included in the analysis. This step was taken to ensure that the data represented the participants’ ability to comprehend inflectional and derivational forms and not their ability to comprehend base forms. Of the 279 participants, the following number incorrectly answered the base form items: use, move, collect (2), circle (6), center (7), accept (24), maintain, develop (11), standard (12), adjust (16), and publish (10). Research questions 1 and 2 were investigated using Cochran’s Q analysis, a nonparametric alternative to an ANOVA which allows for the analysis of multiple repeated measures in the form of dichotomous data. In this study the participants’ performance on the base word and each inflectional and derivational form of that base word were treated as repeated measures. Thus, a significant difference yielded from the dichotomous scores among the base word use, and the tested inflectional forms (used, using, and have used) and tested derivational forms (useful, useless, reuse, reusable, unusable, usage, and user) would indicate a significant difference in the participants’ ability to comprehend the tested forms. Alpha values used in this study for each individual Cochran’s Q analysis were set at 0.001. Research question 3 was investigated using McNemar chi-square analysis, which can be conceptualized as post hoc ANOVA analysis, and is a nonparametric analysis that allows for the analysis of two repeated measures with dichotomous data. To avoid a type 1 error, the alpha values for the McNemar chi-square analysis are dependent on the number of comparisons made between the base word and other members of the same word family. The alpha value was arrived at by dividing 0.05 by the number of comparisons. Cochran’s Q and McNemar chi-square analysis have two assumptions. The first assumption, that data scores are independent of each other, is met. Second, accurate analysis is possible when sample sizes are large, or above 30 participants for Cochran’s Q analysis, while a sample of 26 is sufficient for accurate McNemar chi-square analysis (Green and Salkind 2013). The small number of advanced-group participants (n = 17), and, in contrast, the large size of the entire data set (N = 279), which facilitates small differences in the participants’ ability to comprehend the base form and associated forms yielding statistical significance, means that inferences should be supported by effect-size values. This study adopts Brown and Coombe’s (2015) recommendations for small, medium, and large eta squared (η2) effect sizes of 0.02, 0.13, and 0.26 for Cochran’s Q analysis. The effect-size criteria for the McNemar (φ) analysis are 0.1 (small), 0.3 (medium), 0.5 (large), and 0.8 (very large) (Rovai et al. 2013). RESULTS Table 1 shows the number of participants who correctly answered the base form, and then associated inflectional and derivational forms for each of the 12 target words. Table 1: Number of participants who correctly answered base forms, and inflectional and derivational forms Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle  Adjust  Publish  Base form  277  277  277  272  279  255  268  268  267  273  263  269  -ed  275  277  276  251  275  252  265  261    263  260  266  -ing  276  274  271  240  276  250  261  261    255  258  265  -ing adjective                245          have-ed  270  271  270  237  272  248  260  261    256  260  265  -er  271        276      225      236  265  -able  221    222                194    -less  164                        un-able                150        158  -erly          141                -ingly    139                      un-ing            105              un-ed              89      115      non-                  68        -ment    232            206      194    -ful  200                        -ized                  118        -mentally                77          -ability          62  54  65          76  -ally        215                  -ance            159  185            mis-ed          166              112  anti-ment                80          -alized        74                  -age  63                        counter-    44                      sub-                  28        mal-ed                      19    semi-                    18      en-ment                    16      re-  243          190    225    206  203    -tion      234                    re-able  202                        re-ing                        164  -ic        115                  pre-ed          131                -ist        162                  -ization                  109        -ivization      57                    Noun form                    273      Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle  Adjust  Publish  Base form  277  277  277  272  279  255  268  268  267  273  263  269  -ed  275  277  276  251  275  252  265  261    263  260  266  -ing  276  274  271  240  276  250  261  261    255  258  265  -ing adjective                245          have-ed  270  271  270  237  272  248  260  261    256  260  265  -er  271        276      225      236  265  -able  221    222                194    -less  164                        un-able                150        158  -erly          141                -ingly    139                      un-ing            105              un-ed              89      115      non-                  68        -ment    232            206      194    -ful  200                        -ized                  118        -mentally                77          -ability          62  54  65          76  -ally        215                  -ance            159  185            mis-ed          166              112  anti-ment                80          -alized        74                  -age  63                        counter-    44                      sub-                  28        mal-ed                      19    semi-                    18      en-ment                    16      re-  243          190    225    206  203    -tion      234                    re-able  202                        re-ing                        164  -ic        115                  pre-ed          131                -ist        162                  -ization                  109        -ivization      57                    Noun form                    273      Table 1: Number of participants who correctly answered base forms, and inflectional and derivational forms Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle  Adjust  Publish  Base form  277  277  277  272  279  255  268  268  267  273  263  269  -ed  275  277  276  251  275  252  265  261    263  260  266  -ing  276  274  271  240  276  250  261  261    255  258  265  -ing adjective                245          have-ed  270  271  270  237  272  248  260  261    256  260  265  -er  271        276      225      236  265  -able  221    222                194    -less  164                        un-able                150        158  -erly          141                -ingly    139                      un-ing            105              un-ed              89      115      non-                  68        -ment    232            206      194    -ful  200                        -ized                  118        -mentally                77          -ability          62  54  65          76  -ally        215                  -ance            159  185            mis-ed          166              112  anti-ment                80          -alized        74                  -age  63                        counter-    44                      sub-                  28        mal-ed                      19    semi-                    18      en-ment                    16      re-  243          190    225    206  203    -tion      234                    re-able  202                        re-ing                        164  -ic        115                  pre-ed          131                -ist        162                  -ization                  109        -ivization      57                    Noun form                    273      Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle  Adjust  Publish  Base form  277  277  277  272  279  255  268  268  267  273  263  269  -ed  275  277  276  251  275  252  265  261    263  260  266  -ing  276  274  271  240  276  250  261  261    255  258  265  -ing adjective                245          have-ed  270  271  270  237  272  248  260  261    256  260  265  -er  271        276      225      236  265  -able  221    222                194    -less  164                        un-able                150        158  -erly          141                -ingly    139                      un-ing            105              un-ed              89      115      non-                  68        -ment    232            206      194    -ful  200                        -ized                  118        -mentally                77          -ability          62  54  65          76  -ally        215                  -ance            159  185            mis-ed          166              112  anti-ment                80          -alized        74                  -age  63                        counter-    44                      sub-                  28        mal-ed                      19    semi-                    18      en-ment                    16      re-  243          190    225    206  203    -tion      234                    re-able  202                        re-ing                        164  -ic        115                  pre-ed          131                -ist        162                  -ization                  109        -ivization      57                    Noun form                    273      Research question 1 asked if there is a significant difference in Japanese EFL learners’ ability to comprehend the base form, inflectional forms, and derivational forms. Considering the very high threshold of 98 per cent knowledge of tokens within a text necessary for reading comprehension, the overestimation of learners’ ability to comprehend inflectional and derivational forms is a greater concern than its underestimation. Thus, for WF6 to be recommended as an appropriate general counting unit, a significant difference between the participants’ ability to comprehend the base form, inflectional forms, and derivational forms should not be present. The absence of a significant difference would suggest that the participants’ ability to comprehend the base word is similar to that of the tested inflectional and derivational forms of that base word, and would support the assumption made when using WF6 that comprehension of the base word is evidence of being able to comprehend all other inflectional and derivational forms belonging to that WF6. This is why, when creating WF6 word lists, the presence of derivational forms, for example reusable, is added to the frequency of the use WF6 in that corpus. Likewise, test-takers are accredited with knowledge of unseeingly, if they correctly demonstrate knowledge of see on the WF6-based VST. Or when the lexical load of a text is estimated with a WF6-based lexical profiler, the derived forms of centralization and centric among other derivational forms of center are counted as high-frequency words from the first 1,000 WF6. In contrast, if a significant difference is present, it indicates that the participants’ ability to comprehend a base word is significantly different to that of tested inflectional and derivational forms of that base word, and would not support the assumption made when using WF6. The Cochran’s Q analysis indicated a significant difference (p < 0.001) among the number of correct responses to all 12 base words, and correct responses to associated inflectional and derivational forms (Table 2). The large effect sizes (η2 = 0.36–0.71) of these differences are presented in Table 2. This pattern of significant differences among the number of correct responses to the base word and correct responses to inflectional and derivational forms of the same WF6 is present across all 12 of the target word families among the beginner-group (η2 = 0.44–0.71), intermediate-group (η2 = 0.34–0.73), and advanced-group (η2 = 0.23–0.84) participants, with the exception of the advanced-group participants’ performance on the publish WF6 forms (p = 0.001, η2 = 0.19). The statistically significant discrepancies between the participants’ ability to comprehend the base word and other members of the same WF6 accompanied by large effect sizes (Tables 1 and 2) strongly indicate that WF6 overestimates the participants’ ability to comprehend inflectional and derivational forms. Thus, considering the high threshold of 98 per cent knowledge of tokens within a text necessary for unassisted reading comprehension, WF6 is not the most appropriate general written receptive word counting unit for Japanese EFL learners. Table 2: The significance and effect size of differences in the number of participants who comprehend base forms and the number of participants who comprehend associated WF6 members WF6  All participants  Beginner group  Medium group  Advanced group  Use  N = 277, p = 0.000, η2 = 0.39  n = 84, p = 0.000, η2 = 0.46  n = 176, p = 0.000, η2 = 0.40  n = 17, p = 0.000, η2 = 0.28  Move  N = 277, p = 0.000, η2 = 0.71  n = 84, p = 0.000, η2 = 0.74  n = 176, p = 0.000, η2 = 0.73  n = 17, p = 0.000, η2 = 0.63  Collect  N = 277, p = 0.000, η2 = 0.55  n = 84, p = 0.000, η2 = 0.55  n = 176, p = 0.000, η2 = 0.58  n = 17, p = 0.000, η2 = 0.29  Center  N = 272, p = 0.000, η2 = 0.36  n = 82, p = 0.000, η2 = 0.44  n = 173, p = 0.000, η2 = 0.34  n = 17, p = 0.000, η2 = 0.27  Teach  N = 279, p = 0.000, η2 = 0.48  n = 85, p = 0.000, η2 = 0.56  n = 177, p = 0.000, η2 = 0.48  n = 17, p = 0.000, η2 = 0.24  Accept  N = 255, p = 0.000, η2 = 0.47  n = 70, p = 0.000, η2 = 0.57  n = 168, p = 0.000, η2 = 0.47  n = 17, p = 0.000, η2 = 0.23  Maintain  N = 268, p = 0.000, η2 = 0.55  n = 74, p = 0.000, η2 = 0.67  n = 177, p = 0.000, η2 = 0.54  n = 17, p = 0.000, η2 = 0.30  Develop  N = 268, p = 0.000, η2 = 0.41  n = 77, p = 0.000, η2 = 0.46  n = 174, p = 0.000, η2 = 0.41  n = 17, p = 0.000, η2 = 0.27  Standard  N = 267, p = 0.000, η2 = 0.46  n = 78, p = 0.000, η2 = 0.55  n = 172, p = 0.000, η2 = 0.46  n = 17, p = 0.000, η2 = 0.41  Circle  N = 273, p = 0.000, η2 = 0.64  n = 83, p = 0.000, η2 = 0.71  n = 173, p = 0.000, η2 = 0.70  n = 17, p = 0.000, η2 = 0.84  Adjust  N = 263, p = 0.000, η2 = 0.54  n = 75, p = 0.000, η2 = 0.50  n = 171, p = 0.000, η2 = 0.55  n = 17, p = 0.000, η2 = 0.72  Publish  N = 269, p = 0.000, η2 = 0.46  n = 81, p = 0.000, η2 = 0.59  n = 171, p = 0.000, η2 = 0.44  n = 17, p = 0.001, η2 = 0.19  WF6  All participants  Beginner group  Medium group  Advanced group  Use  N = 277, p = 0.000, η2 = 0.39  n = 84, p = 0.000, η2 = 0.46  n = 176, p = 0.000, η2 = 0.40  n = 17, p = 0.000, η2 = 0.28  Move  N = 277, p = 0.000, η2 = 0.71  n = 84, p = 0.000, η2 = 0.74  n = 176, p = 0.000, η2 = 0.73  n = 17, p = 0.000, η2 = 0.63  Collect  N = 277, p = 0.000, η2 = 0.55  n = 84, p = 0.000, η2 = 0.55  n = 176, p = 0.000, η2 = 0.58  n = 17, p = 0.000, η2 = 0.29  Center  N = 272, p = 0.000, η2 = 0.36  n = 82, p = 0.000, η2 = 0.44  n = 173, p = 0.000, η2 = 0.34  n = 17, p = 0.000, η2 = 0.27  Teach  N = 279, p = 0.000, η2 = 0.48  n = 85, p = 0.000, η2 = 0.56  n = 177, p = 0.000, η2 = 0.48  n = 17, p = 0.000, η2 = 0.24  Accept  N = 255, p = 0.000, η2 = 0.47  n = 70, p = 0.000, η2 = 0.57  n = 168, p = 0.000, η2 = 0.47  n = 17, p = 0.000, η2 = 0.23  Maintain  N = 268, p = 0.000, η2 = 0.55  n = 74, p = 0.000, η2 = 0.67  n = 177, p = 0.000, η2 = 0.54  n = 17, p = 0.000, η2 = 0.30  Develop  N = 268, p = 0.000, η2 = 0.41  n = 77, p = 0.000, η2 = 0.46  n = 174, p = 0.000, η2 = 0.41  n = 17, p = 0.000, η2 = 0.27  Standard  N = 267, p = 0.000, η2 = 0.46  n = 78, p = 0.000, η2 = 0.55  n = 172, p = 0.000, η2 = 0.46  n = 17, p = 0.000, η2 = 0.41  Circle  N = 273, p = 0.000, η2 = 0.64  n = 83, p = 0.000, η2 = 0.71  n = 173, p = 0.000, η2 = 0.70  n = 17, p = 0.000, η2 = 0.84  Adjust  N = 263, p = 0.000, η2 = 0.54  n = 75, p = 0.000, η2 = 0.50  n = 171, p = 0.000, η2 = 0.55  n = 17, p = 0.000, η2 = 0.72  Publish  N = 269, p = 0.000, η2 = 0.46  n = 81, p = 0.000, η2 = 0.59  n = 171, p = 0.000, η2 = 0.44  n = 17, p = 0.001, η2 = 0.19  Table 2: The significance and effect size of differences in the number of participants who comprehend base forms and the number of participants who comprehend associated WF6 members WF6  All participants  Beginner group  Medium group  Advanced group  Use  N = 277, p = 0.000, η2 = 0.39  n = 84, p = 0.000, η2 = 0.46  n = 176, p = 0.000, η2 = 0.40  n = 17, p = 0.000, η2 = 0.28  Move  N = 277, p = 0.000, η2 = 0.71  n = 84, p = 0.000, η2 = 0.74  n = 176, p = 0.000, η2 = 0.73  n = 17, p = 0.000, η2 = 0.63  Collect  N = 277, p = 0.000, η2 = 0.55  n = 84, p = 0.000, η2 = 0.55  n = 176, p = 0.000, η2 = 0.58  n = 17, p = 0.000, η2 = 0.29  Center  N = 272, p = 0.000, η2 = 0.36  n = 82, p = 0.000, η2 = 0.44  n = 173, p = 0.000, η2 = 0.34  n = 17, p = 0.000, η2 = 0.27  Teach  N = 279, p = 0.000, η2 = 0.48  n = 85, p = 0.000, η2 = 0.56  n = 177, p = 0.000, η2 = 0.48  n = 17, p = 0.000, η2 = 0.24  Accept  N = 255, p = 0.000, η2 = 0.47  n = 70, p = 0.000, η2 = 0.57  n = 168, p = 0.000, η2 = 0.47  n = 17, p = 0.000, η2 = 0.23  Maintain  N = 268, p = 0.000, η2 = 0.55  n = 74, p = 0.000, η2 = 0.67  n = 177, p = 0.000, η2 = 0.54  n = 17, p = 0.000, η2 = 0.30  Develop  N = 268, p = 0.000, η2 = 0.41  n = 77, p = 0.000, η2 = 0.46  n = 174, p = 0.000, η2 = 0.41  n = 17, p = 0.000, η2 = 0.27  Standard  N = 267, p = 0.000, η2 = 0.46  n = 78, p = 0.000, η2 = 0.55  n = 172, p = 0.000, η2 = 0.46  n = 17, p = 0.000, η2 = 0.41  Circle  N = 273, p = 0.000, η2 = 0.64  n = 83, p = 0.000, η2 = 0.71  n = 173, p = 0.000, η2 = 0.70  n = 17, p = 0.000, η2 = 0.84  Adjust  N = 263, p = 0.000, η2 = 0.54  n = 75, p = 0.000, η2 = 0.50  n = 171, p = 0.000, η2 = 0.55  n = 17, p = 0.000, η2 = 0.72  Publish  N = 269, p = 0.000, η2 = 0.46  n = 81, p = 0.000, η2 = 0.59  n = 171, p = 0.000, η2 = 0.44  n = 17, p = 0.001, η2 = 0.19  WF6  All participants  Beginner group  Medium group  Advanced group  Use  N = 277, p = 0.000, η2 = 0.39  n = 84, p = 0.000, η2 = 0.46  n = 176, p = 0.000, η2 = 0.40  n = 17, p = 0.000, η2 = 0.28  Move  N = 277, p = 0.000, η2 = 0.71  n = 84, p = 0.000, η2 = 0.74  n = 176, p = 0.000, η2 = 0.73  n = 17, p = 0.000, η2 = 0.63  Collect  N = 277, p = 0.000, η2 = 0.55  n = 84, p = 0.000, η2 = 0.55  n = 176, p = 0.000, η2 = 0.58  n = 17, p = 0.000, η2 = 0.29  Center  N = 272, p = 0.000, η2 = 0.36  n = 82, p = 0.000, η2 = 0.44  n = 173, p = 0.000, η2 = 0.34  n = 17, p = 0.000, η2 = 0.27  Teach  N = 279, p = 0.000, η2 = 0.48  n = 85, p = 0.000, η2 = 0.56  n = 177, p = 0.000, η2 = 0.48  n = 17, p = 0.000, η2 = 0.24  Accept  N = 255, p = 0.000, η2 = 0.47  n = 70, p = 0.000, η2 = 0.57  n = 168, p = 0.000, η2 = 0.47  n = 17, p = 0.000, η2 = 0.23  Maintain  N = 268, p = 0.000, η2 = 0.55  n = 74, p = 0.000, η2 = 0.67  n = 177, p = 0.000, η2 = 0.54  n = 17, p = 0.000, η2 = 0.30  Develop  N = 268, p = 0.000, η2 = 0.41  n = 77, p = 0.000, η2 = 0.46  n = 174, p = 0.000, η2 = 0.41  n = 17, p = 0.000, η2 = 0.27  Standard  N = 267, p = 0.000, η2 = 0.46  n = 78, p = 0.000, η2 = 0.55  n = 172, p = 0.000, η2 = 0.46  n = 17, p = 0.000, η2 = 0.41  Circle  N = 273, p = 0.000, η2 = 0.64  n = 83, p = 0.000, η2 = 0.71  n = 173, p = 0.000, η2 = 0.70  n = 17, p = 0.000, η2 = 0.84  Adjust  N = 263, p = 0.000, η2 = 0.54  n = 75, p = 0.000, η2 = 0.50  n = 171, p = 0.000, η2 = 0.55  n = 17, p = 0.000, η2 = 0.72  Publish  N = 269, p = 0.000, η2 = 0.46  n = 81, p = 0.000, η2 = 0.59  n = 171, p = 0.000, η2 = 0.44  n = 17, p = 0.001, η2 = 0.19  For the flemma to be recommended, a significant difference between the participants’ ability to comprehend the base form and inflectional forms should not be present. Thus, research question 2 asks if there is a significant difference in Japanese EFL learners’ ability to comprehend the base form and associated inflectional forms, and is investigated using Cochran’s Q analysis. Significant differences (p < 0.001) accompanied by small effect sizes were only found among the number of correct responses to three base words (center, η2 = 0.09; develop, η2 = 0.02; circle, and η2 = 0.05) and associated inflectional forms (Table 3). However, a nonsignificant difference is present for eight (use, move, collect, teach, accept, maintain, adjust, and publish) of the 11 flemmas.3 Table 3: The significance and effect size of differences in the number of participants who comprehend base forms and the number of participants who comprehend associated flemma members Flemma  All participants  Beginner group  Intermediate group  Advanced group  Use  N = 277, p = 0.006, η2 = 0.01  n = 84, p = 0.137, η2 = 0.02  n = 176, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Move  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.014, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Collect  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.013, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Center  N = 272, p = 0.000, η2 = 0.09  n = 82, p = 0.000, η2 = 0.1  n = 173, p = 0.000, η2 = 0.09  n = 17, p = 1, η2 = 0.00  Teach  N = 279, p = 0.019, η2 = 0.01  n = 85, p = 0.084, η2 = 0.02  n = 177, p = 0.194, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Accept  N = 255, p = 0.008, η2 = 0.02  n = 70, p = 0.300, η2 = 0.02  n = 168, p = 0.034, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Maintain  N = 268, p = 0.003, η2 = 0.02  n = 74, p = 0.004, η2 = 0.06  n = 177, p = 0.340, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Develop  N = 268, p = 0.000, η2 = 0.03  n = 77, p = 0.007, η2 = 0.05  n = 174, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Standard          Circle  N = 273, p = 0.000, η2 = 0.05  n = 83, p = 0.000, η2 = 0.09  n = 173, p = 0.000, η2 = 0.03  n = 17, p = 1, η2 = 0.00  Adjust  N = 263, p = 0.017, η2 = 0.01  n = 75, p = 0.029, η2 = 0.04  n = 171, p = 0.112, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Publish  N = 269, p = 0.105, η2 = 0.01  n = 81, p = 0.096, η2 = 0.03  n = 171, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Flemma  All participants  Beginner group  Intermediate group  Advanced group  Use  N = 277, p = 0.006, η2 = 0.01  n = 84, p = 0.137, η2 = 0.02  n = 176, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Move  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.014, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Collect  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.013, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Center  N = 272, p = 0.000, η2 = 0.09  n = 82, p = 0.000, η2 = 0.1  n = 173, p = 0.000, η2 = 0.09  n = 17, p = 1, η2 = 0.00  Teach  N = 279, p = 0.019, η2 = 0.01  n = 85, p = 0.084, η2 = 0.02  n = 177, p = 0.194, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Accept  N = 255, p = 0.008, η2 = 0.02  n = 70, p = 0.300, η2 = 0.02  n = 168, p = 0.034, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Maintain  N = 268, p = 0.003, η2 = 0.02  n = 74, p = 0.004, η2 = 0.06  n = 177, p = 0.340, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Develop  N = 268, p = 0.000, η2 = 0.03  n = 77, p = 0.007, η2 = 0.05  n = 174, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Standard          Circle  N = 273, p = 0.000, η2 = 0.05  n = 83, p = 0.000, η2 = 0.09  n = 173, p = 0.000, η2 = 0.03  n = 17, p = 1, η2 = 0.00  Adjust  N = 263, p = 0.017, η2 = 0.01  n = 75, p = 0.029, η2 = 0.04  n = 171, p = 0.112, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Publish  N = 269, p = 0.105, η2 = 0.01  n = 81, p = 0.096, η2 = 0.03  n = 171, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Table 3: The significance and effect size of differences in the number of participants who comprehend base forms and the number of participants who comprehend associated flemma members Flemma  All participants  Beginner group  Intermediate group  Advanced group  Use  N = 277, p = 0.006, η2 = 0.01  n = 84, p = 0.137, η2 = 0.02  n = 176, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Move  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.014, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Collect  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.013, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Center  N = 272, p = 0.000, η2 = 0.09  n = 82, p = 0.000, η2 = 0.1  n = 173, p = 0.000, η2 = 0.09  n = 17, p = 1, η2 = 0.00  Teach  N = 279, p = 0.019, η2 = 0.01  n = 85, p = 0.084, η2 = 0.02  n = 177, p = 0.194, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Accept  N = 255, p = 0.008, η2 = 0.02  n = 70, p = 0.300, η2 = 0.02  n = 168, p = 0.034, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Maintain  N = 268, p = 0.003, η2 = 0.02  n = 74, p = 0.004, η2 = 0.06  n = 177, p = 0.340, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Develop  N = 268, p = 0.000, η2 = 0.03  n = 77, p = 0.007, η2 = 0.05  n = 174, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Standard          Circle  N = 273, p = 0.000, η2 = 0.05  n = 83, p = 0.000, η2 = 0.09  n = 173, p = 0.000, η2 = 0.03  n = 17, p = 1, η2 = 0.00  Adjust  N = 263, p = 0.017, η2 = 0.01  n = 75, p = 0.029, η2 = 0.04  n = 171, p = 0.112, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Publish  N = 269, p = 0.105, η2 = 0.01  n = 81, p = 0.096, η2 = 0.03  n = 171, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Flemma  All participants  Beginner group  Intermediate group  Advanced group  Use  N = 277, p = 0.006, η2 = 0.01  n = 84, p = 0.137, η2 = 0.02  n = 176, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Move  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.014, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Collect  N = 277, p = 0.005, η2 = 0.02  n = 84, p = 0.013, η2 = 0.04  n = 176, p = 0.392, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Center  N = 272, p = 0.000, η2 = 0.09  n = 82, p = 0.000, η2 = 0.1  n = 173, p = 0.000, η2 = 0.09  n = 17, p = 1, η2 = 0.00  Teach  N = 279, p = 0.019, η2 = 0.01  n = 85, p = 0.084, η2 = 0.02  n = 177, p = 0.194, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Accept  N = 255, p = 0.008, η2 = 0.02  n = 70, p = 0.300, η2 = 0.02  n = 168, p = 0.034, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Maintain  N = 268, p = 0.003, η2 = 0.02  n = 74, p = 0.004, η2 = 0.06  n = 177, p = 0.340, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Develop  N = 268, p = 0.000, η2 = 0.03  n = 77, p = 0.007, η2 = 0.05  n = 174, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Standard          Circle  N = 273, p = 0.000, η2 = 0.05  n = 83, p = 0.000, η2 = 0.09  n = 173, p = 0.000, η2 = 0.03  n = 17, p = 1, η2 = 0.00  Adjust  N = 263, p = 0.017, η2 = 0.01  n = 75, p = 0.029, η2 = 0.04  n = 171, p = 0.112, η2 = 0.01  n = 17, p = 1, η2 = 0.00  Publish  N = 269, p = 0.105, η2 = 0.01  n = 81, p = 0.096, η2 = 0.03  n = 171, p = 0.029, η2 = 0.02  n = 17, p = 1, η2 = 0.00  Among the beginner-group and intermediate-group participants, significant differences (p < 0.001) only existed among the number of correct responses to the base words center and circle, and correct responses to their associated inflectional forms (Table 3). Nonsignificant differences are present for nine (use, move, collect, teach, accept, maintain, develop, adjust, and publish) of the 11 flemmas. Furthermore, the effect sizes of the significant differences are small (η2 = 0.03–η2 = 0.1). Among the advanced-group participants (n = 17), no significant differences existed among the number of correct responses to the base words and correct responses to associated inflectional forms (Table 3). The general absence of statistically significant discrepancies among the participants’ ability to comprehend the base words and associated inflectional forms, and small effect sizes in the cases when statistically significant discrepancies were present indicate that the participants possess the ability to comprehend inflectional forms. Thus, the flemma is an appropriate general word counting unit for Japanese EFL learners. Research question 3 asks, if the flemma were adopted, what inflectional and derivational forms would the flemma overestimate or underestimate participants’ ability to comprehend? Research question 3 is addressed through investigating the difference between the accuracy of responses to base forms and individual associated inflectional and derivational forms using McNemar analysis. Considering the high threshold of 98 per cent knowledge of tokens within a text necessary for reading comprehension, the inclusion of an inflectional or derivational form that is significantly more difficult to comprehend than the base form is defined as overestimating the participants’ ability to comprehend a given form. The exclusion of a derivational form which is not significantly more difficult to comprehend than the base form is considered as underestimating learners’ ability to comprehend a derivational form. Table 4 indicates that, among all 279 participants, the adoption of the flemma would only underestimate the participants’ ability to comprehend the derivational forms user, teacher, and publisher, but not developer (φ 0.16) and adjuster (φ 0.1). The same results were found among beginner-group and intermediate-group participants. Table 4: The significance and effect size of differences in the number of participants who comprehend base forms and the number of participants who comprehend associated inflectional forms and derivational forms Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle (verb)  Adjust  Publish  -ed  0.01  0.0  0.0  0.08*  0.01  0.01  0.01  0.03    0.04*  0.01  0.01  -ing  0.0  0.01  0.02  0.12*  0.01  0.02  0.03  0.03    0.07*  0.02  0.01  -ing adjective                0.09*          have-ed  0.03  0.02  0.03  0.13*  0.03  0.03  0.03  0.03    0.07*  0.01  0.01  -er  0.02        0.01      0.16*      0.1*  0.01  -able  0.2*    0.2*                0.26*    -less  0.41*                        un-able                0.44*        0.41*  -erly          0.49*                -ingly    0.5*                      un-ing            0.59*              un-ed              0.67*      0.58*      non-                  0.74*        -ment    0.16*            0.23*      0.26*    -ful  0.28*                        -ized                  0.56*        -mentally                0.71*          -ability          0.78*  0.79*  0.76*          0.72*  -ally        0.21*                  -ance            0.38*  0.31*            mis-ed          0.41*              0.58*  anti-ment                0.7*          -alized        0.73*                  -age  0.77*                        counter-    0.84*                      sub-                  0.89*        mal-ed                      0.93*    semi-                    0.93*      en-ment                    0.94*      re-  0.12*          0.25*    0.16*    0.25*  0.23*    -tion      0.16*                    re-able  0.27*                        re-ing                        0.39*  -ic        0.58*                  pre-ed          0.53*                -ist        0.42*                  -ization                  0.59*        -ivization      0.79*                    Circle (noun)                    0.0      Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle (verb)  Adjust  Publish  -ed  0.01  0.0  0.0  0.08*  0.01  0.01  0.01  0.03    0.04*  0.01  0.01  -ing  0.0  0.01  0.02  0.12*  0.01  0.02  0.03  0.03    0.07*  0.02  0.01  -ing adjective                0.09*          have-ed  0.03  0.02  0.03  0.13*  0.03  0.03  0.03  0.03    0.07*  0.01  0.01  -er  0.02        0.01      0.16*      0.1*  0.01  -able  0.2*    0.2*                0.26*    -less  0.41*                        un-able                0.44*        0.41*  -erly          0.49*                -ingly    0.5*                      un-ing            0.59*              un-ed              0.67*      0.58*      non-                  0.74*        -ment    0.16*            0.23*      0.26*    -ful  0.28*                        -ized                  0.56*        -mentally                0.71*          -ability          0.78*  0.79*  0.76*          0.72*  -ally        0.21*                  -ance            0.38*  0.31*            mis-ed          0.41*              0.58*  anti-ment                0.7*          -alized        0.73*                  -age  0.77*                        counter-    0.84*                      sub-                  0.89*        mal-ed                      0.93*    semi-                    0.93*      en-ment                    0.94*      re-  0.12*          0.25*    0.16*    0.25*  0.23*    -tion      0.16*                    re-able  0.27*                        re-ing                        0.39*  -ic        0.58*                  pre-ed          0.53*                -ist        0.42*                  -ization                  0.59*        -ivization      0.79*                    Circle (noun)                    0.0      Note. Effect sizes (φ) of differences between the number of participants who comprehend base words and associated inflected or derived forms. * significant difference. The alpha values for comparisons were established using the Bonferroni adjustment. Table 4: The significance and effect size of differences in the number of participants who comprehend base forms and the number of participants who comprehend associated inflectional forms and derivational forms Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle (verb)  Adjust  Publish  -ed  0.01  0.0  0.0  0.08*  0.01  0.01  0.01  0.03    0.04*  0.01  0.01  -ing  0.0  0.01  0.02  0.12*  0.01  0.02  0.03  0.03    0.07*  0.02  0.01  -ing adjective                0.09*          have-ed  0.03  0.02  0.03  0.13*  0.03  0.03  0.03  0.03    0.07*  0.01  0.01  -er  0.02        0.01      0.16*      0.1*  0.01  -able  0.2*    0.2*                0.26*    -less  0.41*                        un-able                0.44*        0.41*  -erly          0.49*                -ingly    0.5*                      un-ing            0.59*              un-ed              0.67*      0.58*      non-                  0.74*        -ment    0.16*            0.23*      0.26*    -ful  0.28*                        -ized                  0.56*        -mentally                0.71*          -ability          0.78*  0.79*  0.76*          0.72*  -ally        0.21*                  -ance            0.38*  0.31*            mis-ed          0.41*              0.58*  anti-ment                0.7*          -alized        0.73*                  -age  0.77*                        counter-    0.84*                      sub-                  0.89*        mal-ed                      0.93*    semi-                    0.93*      en-ment                    0.94*      re-  0.12*          0.25*    0.16*    0.25*  0.23*    -tion      0.16*                    re-able  0.27*                        re-ing                        0.39*  -ic        0.58*                  pre-ed          0.53*                -ist        0.42*                  -ization                  0.59*        -ivization      0.79*                    Circle (noun)                    0.0      Word form  Use  Move  Collect  Center  Teach  Accept  Maintain  Develop  Standard  Circle (verb)  Adjust  Publish  -ed  0.01  0.0  0.0  0.08*  0.01  0.01  0.01  0.03    0.04*  0.01  0.01  -ing  0.0  0.01  0.02  0.12*  0.01  0.02  0.03  0.03    0.07*  0.02  0.01  -ing adjective                0.09*          have-ed  0.03  0.02  0.03  0.13*  0.03  0.03  0.03  0.03    0.07*  0.01  0.01  -er  0.02        0.01      0.16*      0.1*  0.01  -able  0.2*    0.2*                0.26*    -less  0.41*                        un-able                0.44*        0.41*  -erly          0.49*                -ingly    0.5*                      un-ing            0.59*              un-ed              0.67*      0.58*      non-                  0.74*        -ment    0.16*            0.23*      0.26*    -ful  0.28*                        -ized                  0.56*        -mentally                0.71*          -ability          0.78*  0.79*  0.76*          0.72*  -ally        0.21*                  -ance            0.38*  0.31*            mis-ed          0.41*              0.58*  anti-ment                0.7*          -alized        0.73*                  -age  0.77*                        counter-    0.84*                      sub-                  0.89*        mal-ed                      0.93*    semi-                    0.93*      en-ment                    0.94*      re-  0.12*          0.25*    0.16*    0.25*  0.23*    -tion      0.16*                    re-able  0.27*                        re-ing                        0.39*  -ic        0.58*                  pre-ed          0.53*                -ist        0.42*                  -ization                  0.59*        -ivization      0.79*                    Circle (noun)                    0.0      Note. Effect sizes (φ) of differences between the number of participants who comprehend base words and associated inflected or derived forms. * significant difference. The alpha values for comparisons were established using the Bonferroni adjustment. In contrast, the adoption of the flemma would overestimate the participants’ ability to comprehend inflectional forms when they are of a different part of speech to the part of speech of the most frequently used base form. As explained in the scoring section, for reasons of ecological validity, markers were instructed not to distinguish between parts of speech when marking the base form. Thus, for the base form circle, if participants produced either the Japanese verb or noun form for circle, the item was marked as correct. Significant differences were found between the base form center, and centered (φ, 0.08), centering (φ 0.12), and have centered (φ. 13); between the base form circle, and circled (φ 0.04), circling (φ 0.07), have circled (φ 0.07); and between develop and developing (the adjectival form) (φ 0.09). This same pattern with almost identical effect sizes was repeated among the beginner-group and intermediate-group participants. Despite the significant difference, the negligible (<0.1) or small (≥0.1) effect sizes indicate that the difference between the adoption of either the lemma or the flemma is minimal. Among the advanced-group participants, the adoption of the flemma does not overestimate the ability of any participant's to comprehend inflectional forms. The small number (n = 17) of advanced-group participants limits the possibility of a significant difference being found between participants’ ability to comprehend base forms and associated derivational forms. Thus, when investigating this issue, the criterion for a significant difference is a small effect size (φ = >0.1), or an accuracy rate of < 88 per cent. Of the 51 derivational forms, 32 derivational forms were found to be significantly more difficult for advanced-group participants to comprehend than the base form. With three exceptions, all of these derivational forms contain multiple affixes, or include affixes described as regular but infrequent affixes (Level 5 derivational forms) or frequent but irregular affixes (Level 6 derivational forms) by Bauer and Nation (1993). This indicates that if the flemma is adopted as a general word counting unit, it would underestimate the advanced-group participants’ ability to comprehend 19 of the 52 tested derivational forms. DISCUSSION Research question 1 asked if there is a significant difference in Japanese EFL learners’ ability to comprehend the base form, inflectional forms, and derivational forms, while research question 2 asked the same question, but only in relation to inflectional forms. Tables 1–4 indicate that, despite demonstrating comprehension of the base word, the participants’ ability to comprehend associated derivational forms was very much incomplete, and was even the case among advanced-group participants. Research question 3 investigated the degree of overestimation and underestimation of the participants' ability to comprehend inflectional forms and derivational forms which would result from the adoption of the flemma. It was found that only slight overestimation would result from the adoption of the flemma among all 279 participants, and beginner-group and medium-group participants. Furthermore, even among advanced-group participants, the adoption of the flemma would only underestimate their ability to comprehend 19 of the 52 tested derivational forms. Thus, this study suggests that WF6 is not an appropriate general written receptive word counting unit for Japanese EFL learners when reading, and that the flemma is a more appropriate general written receptive word counting unit for Japanese EFL learners. Thus, WF6 lexical written receptive knowledge estimates yielded from publically available tests, such as the VST and the NVLT, are overestimations when the participants in the present study are representative of other L1 EFL populations. This is because WF6 based vocabulary tests present examinees with the base form or most common form of the target word and not less commonly known derivational forms, and then if the target form is answered correctly, knowledge of the target form and all associated WF6 members is assumed. However, even if multiple-choice tests presented examinees with derivational forms, instead of base forms, overestimation might persist. This is because within the multiple-choice question stem examinees could potentially find the base form of a word within its derived form without knowing the meaning of the derived form, and then match the base word meaning with the correct multiple-choice answer. For example, in the following modified first 1,000-WF6 VST item, standard is replaced with standardization. An examinee who knows the meaning of standard and normal, but not the meaning of standardization, can find the base word standard within standardization, and then can correctly answer the multiple-choice item below. The test administrator might then wrongly assume that the examinee knows the derived form standardization and other derived -ize and -ization forms. Standardization: Standardization is sometimes a problem. A part of a shoe Marks at school Money people ask for The setting of normal levelsImplications for the accurate measurement of vocabulary knowledge. The accurate measurement of vocabulary size and levels can be addressed through three methods. First, researchers may try to establish what inflectional and derivational forms their participants know before creating appropriate word lists and vocabulary tests. To the best of my knowledge, no researchers have done this, and this is impractical because (i) the ability to comprehend inflectional and derivational forms was found to be base word dependent, (ii) extrapolating knowledge of inflectional and derivational forms containing only one affix to words containing multiple affixes is problematic (17 participants who correctly produced the meaning of both reuse and usable could not produce the meaning of reusable), and (iii) the use of different counting units within a single study inhibits the comparison of results among participants and between studies. Nation and Coxhead (2014) state that prior to Bauer and Nation (1993), it was clear that a standard word family unit was needed, to enable consistency between different word families, and the failure to do so was a contributing factor which renders most twentieth-century research on vocabulary size and levels virtually useless or grossly misleading. A second possible solution is the use of a word counting unit including derivational forms known by most Japanese EFL learners. This study found that only some, not all, derivational -er forms were not significantly more difficult to comprehend than the associated base form. Thus, simply adopting the flemma would seem pragmatic. A third solution is to adopt a smaller counting unit, possibly the flemma for Japanese EFL learners. The adoption of the flemma has various other advantages. First, the flemma is an easily understood unit allowing readers to easily comprehend and evaluate research findings, thus, making the replication and comparison of studies more feasible (Schmitt 2010). In contrast, WF6 is more problematic than other counting units because it is unclear which forms belong in the WF6 unit. What about infrequent and hyphenated forms such as teacher-like or developmentwise? Second, to facilitate the comparison of receptive and productive studies, the flemma is a reasonable compromise. Nation (2006b), Schmitt (2010), Nation and Webb (2011), and Nation (2013) have argued that the WF6 unit should not be used when measuring productive vocabulary knowledge. Implications for research The validity of the research paradigm that short corpora-derived lists of high-frequency WF6 lists should be deliberately taught and learned because they cover such a large percentage of the tokens that English language learners will actually encounter is questioned by this study. Similarly, this study questions the validity of studies which cite Hu and Nation (2000) when estimating the WF6 vocabulary level necessary to comprehend texts. This is because these studies are based on the coverage provided by WF6 and not the flemma, and thus assume that learners are able to comprehend all WF6 forms. Thus, the results of this study argue that inferences made in studies based on the coverage of WF6 are of limited validity unless evidence exists that participants and learners are able to comprehend WF6 inflectional and derivational forms. This argument is supported by the figures in Brown (2017) which show that text from the BNC which has 98 per cent coverage on the basis of WF6 has only 85 per cent coverage on the basis of the flemma, which this study indicates is an appropriate word counting unit for Japanese EFL learners. LIMITATIONS The above results should be viewed in the context of the limitations of this study. Although the participants in this study were from a wide range of proficiencies, they are all L1 Japanese EFL learners. Thus, generalizing the results of this study to learners of different proficiency levels or L1s, particularly Latin- or Germanic-based languages is problematic. Second, it would have been preferable to test a larger range of inflectional and derivational forms. Third, the participants’ ability to comprehend derivational and inflectional forms of only 12 words was investigated. This is an issue because this study found that the degree to which participants comprehend inflectional forms and derivational forms consisting of the same affix was base word dependent. Fourth, while this study refers to the decreases in the coverage provided by the flemma relative to the word family in the BNC, the degree to which WF6 and the flemma provide varying coverage of a range of individual texts of varying difficulties and types is unknown, and it is hoped that future research investigates this. Finally, the findings of this study are limited to the comprehension of orthographic forms when reading, and so further research into an appropriate counting unit when listening, when speaking, and when writing is necessary. CONCLUSION This article adds to the existing L2 literature on EFL learners’ knowledge of inflectional and derivational forms by directly addressing the question of the appropriateness of WF6 and the flemma as a written receptive word counting unit with Japanese EFL learners. Results indicate that despite comprehending base words, the participants’ ability to comprehend associated derivational forms was very limited. Thus, the flemma was found to be more appropriate than WF6 as a word counting unit for these participants, including those who demonstrated mastery of the base forms of the fourth or fifth 1,000 BNC/COCA WF6 bands. Even among the advanced-group participants, considering the 98 per cent threshold for reading comprehension, it is not possible to recommend the adoption of WF6. Finally, results indicate that participants’ ability to comprehend inflectional and derivational forms was base word dependent. Thus, when diagnosing learner’s knowledge of affixes, inferences should be made with caution. These findings, combined with those of Schmitt and Meara (1997), Mochizuki and Aizawa (2000), Sasao and Webb (2015), and Ward and Chuenjundaeng (2009) question the validity of the research and vocabulary tests that have adopted WF6, the use of WF6 word lists, and the research paradigm that short corpora-derived lists of high-frequency words should be deliberately taught and learned because they cover such a large percentage of the tokens that English language learners will encounter. SUPPLEMENTARY DATA Supplementary material is available at Applied Linguistics online. Acknowledgements The author would like to thank the editor and reviewers whose comments and suggestions have greatly improved this article. The author would also like to thank Paul Nation, Batia Laufer, Kiyomi Yoshizawa, Sawako Hamatani, Brandon Kramer, Geoffrey Pinchbeck, Laura Huston, Steve Porritt, Tim Stoeckel, and Dale Brown for their feedback on this article. The author has special thanks for David Beglar who greatly improved the quality of the article. Any remaining faults are the sole responsibility of the author. Funding This work was supported by JSPS KAKENHI (grant number JP 16K16890). Conflict of interest statement. None declared. NOTES ON CONTRIBUTOR Stuart McLean is an instructor at Osaka Jogakuin University. He holds an M.S.Ed. (TESOL), a Ph.D. in Forensic Medicine, and a P.G.C.E, and is an Applied Linguistics doctoral student at Kansai University. He has published in Reading in a Foreign Language, Vocabulary Learning and Instruction, Language Teaching Research, TESOL Quarterly, Applied Linguistics, and Language Assessment Quarterly. Address for correspondence: Osaka Jogakuin University, 2-26-54 Tamatsukuri, Osaka-shi, Osaka, Japan 540-004. <stuart93@me.com> Notes 1 While this study refers to the decreases in the coverage provided by the flemma relative to the word family in the BNC, the degree to which WF6 and the flemma provide varying coverage of a range of individual texts of varying difficulties and types requires investigation. 2 Prior to participants completing the inflectional and derivational forms comprehension test, participants completed a productive test (L1 meaning to L2 form translation test) for the same target items. Participants were not handed the comprehension test until the productive test was completed and returned to the researcher. The sequence of measuring productive and then receptive knowledge of inflectional and derivational forms, the large number of items in each test (K = 100), and the test items being ordered differently in each test ensure that test effect, if present, is minimal (Nation, 2013). Interviews with the participants indicated that no test effect was present. This is an example of a productive test item: 使用可能な(useable): The computer is now ________________ . The translation in parentheses was not visible to examinees. 3 In the 12 target WF6 sets, the standard WF6 set has no inflectional forms. Thus, the number of WF6 sets among which a difference in participants’ ability to comprehend the base forms, and inflectional forms was 11 and not 12. REFERENCES Bauer L. Nation P.. 1993. ‘ Word families,’ International Journal of Lexicography  6: 253– 79. Google Scholar CrossRef Search ADS   Brown D. (2017). Examining the word family through word lists (in preparation). Brown J. D. Coombe C.. 2015. Research in language Teaching and Learning . Cambridge University Press. Francis W. Kucˇera H.. 1982. Frequency analysis of English usage . Houghton Mifflin. Gardner D. 2007. ‘ Validating the construct of word in applied corpus-based vocabulary research—A critical survey,’ Applied Linguistics  28: 241– 65. Google Scholar CrossRef Search ADS   Green B. Salkind J.. 2013. Using SPSS for Windows and Macintosh . Prentice Hall Press. Gyllstad H. Vilkaite L. Schmitt N.. 2015. ‘ Assessing vocabulary size through multiple-choice formats: Issues with guessing and sampling rates,’ ITL International Journal of Applied Linguistics  166: 278– 306. Google Scholar CrossRef Search ADS   Hu M. Nation I. S. P.. 2000. ‘ Vocabulary density and reading comprehension,’ Reading in a Foreign Language  13: 403– 30. Laufer B. 1997. ‘The lexical plight in second language reading: Words you don't know, words you think you know, and words you can't guess’ in Coady J. Huckin T. (eds): Second Language Vocabulary Acquisition: A Rationale for Pedagogy , pp. 20– 34. Cambridge University Press. Laufer B. Goldstein Z.. 2004. ‘ Testing vocabulary knowledge—Size, strength, and computer adaptiveness,’ Language Learning  54: 399– 436. Google Scholar CrossRef Search ADS   Macalister J. Webb S.. 2013. ‘ A response,’ TESOL Quarterly  47: 852– 5. Google Scholar CrossRef Search ADS   Meara P. Buxton B.. 1987. ‘ An alternative to multiple choice vocabulary tests,’ Language Testing  4: 142– 54. Google Scholar CrossRef Search ADS   McLean S. Kramer B.. 2015. ‘ The creation of a new vocabulary levels test,’ Shiken  19: 1– 11. McLean S. Ishii T. Stoeckel T. Bennett P. Matsumoto Y.. 2016. ‘ An edited version of the first eight 1,000-word frequency bands of the Japanese-English version of the vocabulary size test,’ The Language Teacher  40: 3– 7. Mochizuki M. Aizawa K.. 2000. ‘ An affix acquisition order for EFL learners—An exploratory study,’ System  28: 291– 304. Google Scholar CrossRef Search ADS   Nagy W. Diakidoy I.-A. Anderson R.. 1993. ‘ The acquisition of morphology: Learning the contribution of suffixes to the meanings of derivatives,’ Journal of Reading Behavior  25: 155– 80. Google Scholar CrossRef Search ADS   Nation I. S. P. 2001. Learning Vocabulary in Another Language . Cambridge University Press. Google Scholar CrossRef Search ADS   Nation I. S. P. 2006a. BNC-based Word Lists . Victoria University of Wellington. Nation I. S. P. 2006b. ‘ How large a vocabulary is needed for reading and listening?,’ The Canadian Modern Language Review  6: 59– 82. Google Scholar CrossRef Search ADS   Nation I. S. P. 2012. The Vocabulary Size Test. Nation I. S. P. 2013. Learning Vocabulary in Another Language . Cambridge University Press. Google Scholar CrossRef Search ADS   Nation I. S. P. 2014. ‘ How much input do you need to learn the most frequent 9,000 words?,’ Reading in a Foreign Language  26: 1– 16. Nation I. S. P. Beglar D.. 2007. ‘ A vocabulary size test,’ The Language Teacher  31: 9– 13. Nation I. S. P. Coxhead A.. 2014. ‘ Vocabulary size research at Victoria University of Wellington, New Zealand,’ Language Teaching  47: 398– 403. Google Scholar CrossRef Search ADS   Nation I. S. P. Webb S.. 2011. Researching and Analyzing Vocabulary . Heinle. Pinchbeck G. G. 2014. Lexical Frequencies Profiling of Canadian High School Diploma Exam Expository Writing: L1 and L2 Academic English. Roundtable presentation at American Association of Applied Linguistics, Toronto, Ontario. Rovai A. P. Baker J. D. Ponton M. K.. 2013. Social Science Research Design and Statistics: A Practitioner's Guide to Research Methods and IBM SPSS . Watertree Press LLC. Sasao Y. Webb S.. 2015. ‘ The Word Part Levels Test,’ Language Teaching Research  19: 667– 86. Google Scholar CrossRef Search ADS   Schmitt N. 2010. Researching Vocabulary: A Vocabulary Research Manual . Palgrave Press. Google Scholar CrossRef Search ADS   Schmitt N. Cobb T. Horst M. Schmitt D.. 2015. ‘ How much vocabulary is needed to use English? Replication of Van Zeeland & Schmitt (2012), Nation, (2006), and Cobb (2007),’ Language Teaching  1– 15. Schmitt N. Jiang X. Grabe W.. 2011. ‘ The percentage of words known in a text and reading comprehension,’ The Modern Language Journal  95: 26– 43. Google Scholar CrossRef Search ADS   Schmitt N. Schmitt D. Clapham C.. 2001. ‘ Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test,’ Language Testing  18: 55– 88. Google Scholar CrossRef Search ADS   Schmitt N. Meara P.. 1997. ‘ Researching vocabulary through a word knowledge framework,’ Studies in Second Language Acquisition  19: 17– 36. Google Scholar CrossRef Search ADS   Sinclair J. 2004. How to Use Corpora in Language Teaching . John Benjamins Publishing Company. Google Scholar CrossRef Search ADS   Swenson E. West M.. 1934. On the Counting of New Words in Textbooks for Teaching Foreign Languages No. 1 . University of Toronto. Tyler A. Nagy W.. 1989. ‘ The acquisition of English derivational morphology,’ Journal of Memory and Language  28: 649– 67. Google Scholar CrossRef Search ADS   Ward J. Chuenjundaeng J.. 2009. ‘ Suffix knowledge: Acquisition and applications,’ System  37: 461– 9. Google Scholar CrossRef Search ADS   Webb S. 2010. ‘ A corpus driven study of the potential for vocabulary learning through watching movies,’ International Journal Of Corpus Linguistics  15: 497– 519. Google Scholar CrossRef Search ADS   Webb S. A. Sasao Y.. 2013. ‘ New Directions In vocabulary testing,’ RELC Journal  44: 263– 77. Google Scholar CrossRef Search ADS   © Oxford University Press 2017

Journal

Applied LinguisticsOxford University Press

Published: Feb 4, 2017

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off