TY - JOUR
AU - Bennett,, Phil
AB - Word lists have long been used to guide learners’ lexical acquisition, develop course materials, and form a basis for vocabulary tests. This has led to several recent publications discussing the principles of optimal word list development (Nation 2016; Brezina and Gablasova 2017; Stein 2017). A key consideration is the principle by which words are grouped on such lists. That is, if a learner exhibits knowledge of a headword, how many related words can that learner be safely assumed to also know? This question is important, since we need reliable principles to break down the challenge of vocabulary acquisition. Influential word lists, such as Nation’s version of the General Service List that accompanies the Range program (Heatley et al. 1994) and Coxhead’s (2000) Academic Word List, as well as instruments, such as the Vocabulary Size Test (Nation and Beglar 2007), utilize the word family as a grouping principle, with headwords being grouped together with many of their inflections, regular derivations, and irregular but frequent derived forms. However, there is growing evidence that for many second language learners, the word family may not be appropriate for purposes of pedagogy and assessment (Schmitt and Meara 1997; Mochizuki and Aizawa 2000; Gardner 2007; Ward and Chuenjundaeng 2009). Recent research by McLean (2017) found that all but the most advanced learners had limited knowledge of derived forms, leading McLean to argue that the less inclusive modified lemma, or flemma, should be adopted as the basis for word lists for most learners. A regular lemma consists of a headword plus only inflections that are the same part of speech (POS) as the headword, for instance the nominal headword pause and the plural pauses. In a flemma, when a headword has multiple POS, inflections for each POS are included. Thus, the flemma for the headword pause includes the nouns pause and pauses as well as the verbs pause, pauses, paused, and pausing. This grouping assumes that learners can make conceptual links between the use of flemma constituents to express, for example, a nominal entity, a verbal process, or an adjectival attribute. McLean (2017) found that learners with knowledge of a headword were often unable to demonstrate receptive knowledge of associated derivations but usually did have knowledge of inflections that were the same POS as the headword. However, McLean did not sufficiently ascertain whether learners understood flemma constituents which crossed the POS boundary. For example, would learners know pause as a verb if they knew the meaning of pause as a noun? In other words, while McLean provided substantial evidence to support the use of the lemma over the word family, additional research is needed before we can say the flemma is more appropriate than the lemma for grouping words. 1. PURPOSE The present study addressed this issue with the following research question: for words with multiple POS, are learners who have receptive knowledge of the word in one POS also able to understand it in another POS? 2. METHODS 2.1 Participants The convenience sample included 64 L1 Japanese speakers (25 male, 39 female) from four intact English classes at two colleges in Japan. From the first institution were 16 (9 male, 7 female) learners enrolled in one class (institutional TOEIC Listening and Reading Test M = 391.9, SD = 97.0); from the second institution were 48 (16 male, 32 female) students from three classes (Computerized Assessment System for English Communication [CASEC] M = 603.5, SD = 93.8). These TOEIC and CASEC scores are approximately representative of the English proficiency of Japanese university students and correspond with the following distribution of participants across CEFR levels: A1 n = 5, A2 n = 25, B1 n = 31, B2 n = 3 (see Educational Testing Service 2015; Computerized Assessment System 2018). 2.2 Instrument To determine whether participants had receptive knowledge of the meaning of target words with multiple POS, they were asked to translate short, decontextualized sentences, each containing a target word (see Supplementary Materials). We elicited translations of complete sentences because Japanese learners do not always disambiguate POS in translating single words (e.g. they may translate both the noun and verb forms of study as ). Such responses signify partial knowledge, but they do not indicate whether incomplete word knowledge inhibits comprehension. Because sentence-length translations were elicited, two precautions were taken to ensure that the instrument assessed knowledge of the tested words and not grammar or other vocabulary. First, example sentences contained just one clause and, except for target words, used lexis only from the first 1,000 words of English. Second, learners’ knowledge of basic sentence structure was verified during data collection. The instrument elicited knowledge of 12 word forms from the first 3,000 words of English. These target words were selected to represent the average and range of difficulty of the 200 words tested on Forms A and B of the flemma-based New General Service List Test (Stoeckel et al. 2018) administered to 598 Japanese university students of similar proficiency to those in the present study. Each target word has two parts of speech with essentially the same meaning sense. For example, pause basically means ‘a temporary stop’ in both its noun and verb forms. For each target word, two test items were written, one for each POS. The items for pause were: I took a pause here. We usually pause here. To minimize a priming effect, items testing the same word were separated at random intervals, with the lower-frequency POS appearing first.1 Additionally, the online format displayed items one at a time, and revisiting completed questions was not allowed. 2.3 Procedures The test was administered under supervision with no time limit. Students were instructed to translate each sentence or to write ‘I don’t know’ if they had no idea of its meaning. Dichotomous marking was based on judgements of participants' target word knowledge, with mistranslations of other sentence parts being ignored. Importantly, as long as translations demonstrated understanding of target words in the context of the example sentences in which they appeared, indirect or atypical renderings for target words or changes in their POS were not disqualifying. Table 1 illustrates these criteria. Two Japanese native speakers with SLA/TESOL backgrounds marked all responses. Based on Fleiss (1971), inter-rater reliability was ‘excellent’ as indexed by Cohen’s kappa (⁠ κ = 0.787). Divergent ratings were resolved by referring to the Oxford English Dictionary, and subsequent discussion until consensus was reached (Cronbach α = 0.87). Table 1: Example responses and judgments for POS test Prompt Translation Meaning Judgment This compounds the problem. This compound is a problem. NG I have one extra at home. I have an extra house. NG Your action results in three things. Your action brought three things as a result. OK Your action leads to three things. OK Your reaction leads to three results. OK They have variable prices. The price of these things changes. OK Prompt Translation Meaning Judgment This compounds the problem. This compound is a problem. NG I have one extra at home. I have an extra house. NG Your action results in three things. Your action brought three things as a result. OK Your action leads to three things. OK Your reaction leads to three results. OK They have variable prices. The price of these things changes. OK Table 1: Example responses and judgments for POS test Prompt Translation Meaning Judgment This compounds the problem. This compound is a problem. NG I have one extra at home. I have an extra house. NG Your action results in three things. Your action brought three things as a result. OK Your action leads to three things. OK Your reaction leads to three results. OK They have variable prices. The price of these things changes. OK Prompt Translation Meaning Judgment This compounds the problem. This compound is a problem. NG I have one extra at home. I have an extra house. NG Your action results in three things. Your action brought three things as a result. OK Your action leads to three things. OK Your reaction leads to three results. OK They have variable prices. The price of these things changes. OK To quantify the extent to which word knowledge in one POS corresponded with that in the other POS, Jaccard’s index (Jaccard 1912) was calculated for each target word. In the context of the present study, Jaccard’s index represents the proportion of participants with word knowledge in both POS out of those who demonstrated understanding in at least one POS. Jaccard’s index ranges from 0 to 1, with 0 indicating that each learner who has lexical knowledge in one POS has comprehension in only one POS, and 1 indicating that all learners with knowledge in one POS also have knowledge in the other. Values close to 1 would support the use of the flemma. 3. RESULTS For the 12 target words under investigation, Jaccard indices ranged from 0.00 (compound) to 0.82 (edit; Table 2). The combined Jaccard index for all target words was 0.56, indicating that when word knowledge was demonstrated in one POS, it was also exhibited for the other POS 56 per cent of the time. Table 2: Jaccard indices for target words Tested form (POS 1, POS 2) POS knowledge Jaccard index Both POS 1 POS 2 Neither Edit (n, v) 38 3 5 18 0.82 Result (n, v) 45 14 1 4 0.75 Pause (n, v) 33 6 10 15 0.67 Export (n, v) 33 5 16 10 0.61 Quote (n, v) 16 10 3 35 0.55 Rise (n, v) 28 17 7 12 0.54 Fool (n, v) 26 1 22 15 0.53 Extra (adj, n) 23 18 7 16 0.48 Function (n, v) 21 21 5 17 0.45 Twist (n, v) 15 2 21 26 0.39 Variable (adj, n) 5 5 8 46 0.28 Compound (n, v) 0 13 6 45 0.00 Summary 283 115 111 259 0.56 Tested form (POS 1, POS 2) POS knowledge Jaccard index Both POS 1 POS 2 Neither Edit (n, v) 38 3 5 18 0.82 Result (n, v) 45 14 1 4 0.75 Pause (n, v) 33 6 10 15 0.67 Export (n, v) 33 5 16 10 0.61 Quote (n, v) 16 10 3 35 0.55 Rise (n, v) 28 17 7 12 0.54 Fool (n, v) 26 1 22 15 0.53 Extra (adj, n) 23 18 7 16 0.48 Function (n, v) 21 21 5 17 0.45 Twist (n, v) 15 2 21 26 0.39 Variable (adj, n) 5 5 8 46 0.28 Compound (n, v) 0 13 6 45 0.00 Summary 283 115 111 259 0.56 Table 2: Jaccard indices for target words Tested form (POS 1, POS 2) POS knowledge Jaccard index Both POS 1 POS 2 Neither Edit (n, v) 38 3 5 18 0.82 Result (n, v) 45 14 1 4 0.75 Pause (n, v) 33 6 10 15 0.67 Export (n, v) 33 5 16 10 0.61 Quote (n, v) 16 10 3 35 0.55 Rise (n, v) 28 17 7 12 0.54 Fool (n, v) 26 1 22 15 0.53 Extra (adj, n) 23 18 7 16 0.48 Function (n, v) 21 21 5 17 0.45 Twist (n, v) 15 2 21 26 0.39 Variable (adj, n) 5 5 8 46 0.28 Compound (n, v) 0 13 6 45 0.00 Summary 283 115 111 259 0.56 Tested form (POS 1, POS 2) POS knowledge Jaccard index Both POS 1 POS 2 Neither Edit (n, v) 38 3 5 18 0.82 Result (n, v) 45 14 1 4 0.75 Pause (n, v) 33 6 10 15 0.67 Export (n, v) 33 5 16 10 0.61 Quote (n, v) 16 10 3 35 0.55 Rise (n, v) 28 17 7 12 0.54 Fool (n, v) 26 1 22 15 0.53 Extra (adj, n) 23 18 7 16 0.48 Function (n, v) 21 21 5 17 0.45 Twist (n, v) 15 2 21 26 0.39 Variable (adj, n) 5 5 8 46 0.28 Compound (n, v) 0 13 6 45 0.00 Summary 283 115 111 259 0.56 4. DISCUSSION AND CONCLUSION This study, extending the work of McLean (2017), set out to examine which word grouping principle is more appropriate for learners of English, the lemma or the flemma. The feature that distinguishes these two levels is knowledge of orthographically identical words at multiple POS, an aspect of word knowledge that was not sufficiently examined by McLean. We found that for cases in which learners exhibited lexical knowledge in one POS, they understood the word in another POS just 56 per cent of the time, with substantial variation among the 12 words tested. By comparison, in McLean’s study, learners with headword knowledge understood related inflectional forms in the same POS 97 per cent of the time; his argument against the use of the word family was based on the fact that headword knowledge was associated with comprehension of just 54 per cent of derivational forms. Thus, to be consistent with McLean’s approach and conclusions, if we wish to group words such that knowledge of a headword implies knowledge of other constituents, then the lemma ought to be used. While this exploratory study has raised questions over the suitability of flemma-based groupings for learners of low to intermediate-level proficiency, several details require further investigation. Most obviously, the examination needs to be extended to a greater number of words, particularly at the highest frequency levels where forms with multiple POS are most common. It would also be informative to include participants with a wider range of proficiencies. This may yield insights into how lexical knowledge becomes integrated with the morphological and grammatical elements of a powerful vocabulary. Finally, research involving learners from Indo-European and other L1 backgrounds could provide additional insights. If further inquiry supports the current findings, for vocabulary level or size determinations in research involving low- or intermediate-level learners, assessment should be done at the lemma level because flemma-based tests could overestimate lexical knowledge. For pedagogy, flemma-based lists may remain useful, since current vocabulary profiling software does not distinguish orthographically identical word forms. However, if such lists are used with learners, they would be of greater value if POS were explicitly noted for each entry. To sum up, McLean (2017) effectively addressed methodological shortcomings of earlier inquiry to provide compelling evidence against the use of the word family when headword knowledge assumes comprehension of other word group members. The present study extends McLean’s work by considering whether the lemma or flemma accords more with this assumption. Tim Stoeckel is an associate professor at the University of Niigata Prefecture. His primary interests are in vocabulary assessment, vocabulary development, and L2 reading fluency. Address for correspondence: Tim Stoeckel, University of Niigata Prefecture, 471 Ebigase, Higashi-ku, Niigata City, Niigata, 950-8680, Japan. <stoeckel@unii.ac.jp> Tomoko Ishii is an associate professor at Meiji Gakuin University. Her primary research interest is in memory studies and vocabulary development, and she has worked also on vocabulary assessment. Phil Bennett is an associate professor at the University of Niigata Prefecture. His research interests are mainly in vocabulary development, testing, and metaphorical vocabulary acquisition. Note Footnotes 1 Frequencies at each POS were obtained from https://corpus.byu.edu/coca/ References Brezina V. , Gablasova D. . 2017 . ‘ How to produce vocabulary lists? Issues of definition, selection and pedagogical aims. A response to Gabriele Stein ,’ Applied Linguistics 38 : 764 – 7 . Google Scholar Crossref Search ADS Computerized Assessment System for English Communication . October 9 2018 . ‘Data & information,’ available at https://global.casec.com/data/#cefr Coxhead A. 2000 . ‘ A new academic word list ,’ TESOL Quarterly 34 : 213 – 38 . Google Scholar Crossref Search ADS Educational Testing Service . 2015 . ‘Mapping the TOEIC tests on the CEFR,’ available at https://www.ets.org/s/toeic/pdf/toeic_cef_mapping_flyer.pdf Fleiss J. L. 1971 . ‘ Measuring nominal scale agreement among many raters ,’ Psychological Bulletin 76 : 378 – 82 . Google Scholar Crossref Search ADS Gardner D. 2007 . ‘ Validating the construct of word in applied corpus-based vocabulary research: A critical survey ,’ Applied Linguistics 28 : 241 – 65 . Google Scholar Crossref Search ADS Heatley A. , Nation P. , Coxhead A. . 1994 . Range [Computer software], available at https://www.victoria.ac.nz/lals/about/staff/paul-nation Jaccard P. 1912 . ‘ The distribution of the flora in the Alpine zone ,’ The New Phytologist 11 : 37 – 50 . Google Scholar Crossref Search ADS McLean S. 2017 . ‘ Evidence for the adoption of the flemma as an appropriate word counting unit ,’ Applied Linguistics 39 : 823 – 45 . doi: 10.1093/applin/amw050. Google Scholar Crossref Search ADS Mochizuki M. , Aizawa K. . 2000 . ‘ An affix acquisition order for EFL learners: An exploratory study ,’ System 28 : 291 – 304 . Google Scholar Crossref Search ADS Nation I. S. P. 2016 . Making and Using Word Lists for Language Learning and Testing . John Benjamins Publishing Company . Nation P. , Beglar D. . 2007 . ‘ A vocabulary size test ,’ The Language Teacher 31 : 9 – 13 . Schmitt N. , Meara P. . 1997 . ‘ Researching vocabulary through a word knowledge framework: Word associations and verbal suffixes ,’ Studies in Second Language Acquisition 19 : 17 – 36 . Google Scholar Crossref Search ADS Stein G. 2017 . ‘ Some thoughts on the issue of core vocabularies ,’ Applied Linguistics 38 : 759 – 63 . Stoeckel T. , Ishii T. , Bennett P. . 2018 . ‘ A Japanese-English bilingual version of the New General Service List Test ,’ JALT Journal 40 : 5 – 22 . Ward J. , Chuenjundaeng J. . 2009 . ‘ Suffix knowledge: Acquisition and applications ,’ System 37 : 461 – 9 . Google Scholar Crossref Search ADS © The Author(s) (2018). Published by Oxford University Press. All rights reserved. For permissions, please email: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)
TI - Is the Lemma More Appropriate than the Flemma as a Word Counting Unit?
JF - Applied Linguistics
DO - 10.1093/applin/amy059
DA - 2018-12-19
UR - https://www.deepdyve.com/lp/oxford-university-press/is-the-lemma-more-appropriate-than-the-flemma-as-a-word-counting-unit-NBanyFNFK0
SP - 1
VL - Advance Article
IS - 
DP - DeepDyve
ER -