CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON

CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON Even though considerable attention has been given to the polarity of words (positive and negative) and the creation of large polarity lexicons, research in emotion analysis has had to rely on limited and small emotion lexicons. In this paper, we show how the combined strength and wisdom of the crowds can be used to generate a large, high‐quality, word–emotion and word–polarity association lexicon quickly and inexpensively. We enumerate the challenges in emotion annotation in a crowdsourcing scenario and propose solutions to address them. Most notably, in addition to questions about emotions associated with terms, we show how the inclusion of a word choice question can discourage malicious data entry, help to identify instances where the annotator may not be familiar with the target term (allowing us to reject such annotations), and help to obtain annotations at sense level (rather than at word level). We conducted experiments on how to formulate the emotion‐annotation questions, and show that asking if a term is associated with an emotion leads to markedly higher interannotator agreement than that obtained by asking if a term evokes an emotion. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Computational Intelligence Wiley

CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON

Loading next page...
 
/lp/wiley/crowdsourcing-a-word-emotion-association-lexicon-0mncEQuDWH
Publisher
Wiley
Copyright
© 2012 National Research Council Canada
ISSN
0824-7935
eISSN
1467-8640
DOI
10.1111/j.1467-8640.2012.00460.x
Publisher site
See Article on Publisher Site

Abstract

Even though considerable attention has been given to the polarity of words (positive and negative) and the creation of large polarity lexicons, research in emotion analysis has had to rely on limited and small emotion lexicons. In this paper, we show how the combined strength and wisdom of the crowds can be used to generate a large, high‐quality, word–emotion and word–polarity association lexicon quickly and inexpensively. We enumerate the challenges in emotion annotation in a crowdsourcing scenario and propose solutions to address them. Most notably, in addition to questions about emotions associated with terms, we show how the inclusion of a word choice question can discourage malicious data entry, help to identify instances where the annotator may not be familiar with the target term (allowing us to reject such annotations), and help to obtain annotations at sense level (rather than at word level). We conducted experiments on how to formulate the emotion‐annotation questions, and show that asking if a term is associated with an emotion leads to markedly higher interannotator agreement than that obtained by asking if a term evokes an emotion.

Journal

Computational IntelligenceWiley

Published: Aug 1, 2013

References

  • Memose: Search engine for emotions in multimedia documents
    Knautz, Knautz; Siebenlist, Siebenlist; Stock, Stock
  • Spatial presence and emotions during video game playing: Does it matter with whom you play?
    Ravaja, Ravaja; Saari, Saari; Turpeinen, Turpeinen; Laarni, Laarni; Salminen, Salminen; Kivikangas, Kivikangas
  • Online trust: A stakeholder perspective, concepts, implications, and future directions
    Shankar, Shankar; Urban, Urban; Sultan, Sultan
  • Measuring praise and criticism: Inference of semantic orientation from association
    Turney, Turney; Littman, Littman

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create folders to
organize your research

Export folders, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off