Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Exploring the influence of multimodal social media data on stock performance: an empirical perspective and analysis

Exploring the influence of multimodal social media data on stock performance: an empirical... Despite the extensive academic interest in social media sentiment for financial fields, multimodal data in the stock market has been neglected. The purpose of this paper is to explore the influence of multimodal social media data on stock performance, and investigate the underlying mechanism of two forms of social media data, i.e. text and pictures.Design/methodology/approachThis research employs panel vector autoregressive models to quantify the effect of the sentiment derived from two modalities in social media, i.e. text information and picture information. Through the models, the authors examine the short-term and long-term associations between social media sentiment and stock performance, measured by three metrics. Specifically, the authors design an enhanced sentiment analysis method, integrating random walk and word embeddings through Global Vectors for Word Representation (GloVe), to construct a domain-specific lexicon and apply it to textual sentiment analysis. Secondly, the authors exploit a deep learning framework based on convolutional neural networks to analyze the sentiment in picture data.FindingsThe empirical results derived from vector autoregressive models reveal that both measures of the sentiment extracted from textual information and pictorial information in social media are significant leading indicators of stock performance. Moreover, pictorial information and textual information have similar relationships with stock performance.Originality/valueTo the best of the authors’ knowledge, this is the first study that incorporates multimodal social media data for sentiment analysis, which is valuable in understanding pictures of social media data. The study offers significant implications for researchers and practitioners. This research informs researchers on the attention of multimodal social media data. The study’s findings provide some managerial recommendations, e.g. watching not only words but also pictures in social media. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Internet Research Emerald Publishing

Exploring the influence of multimodal social media data on stock performance: an empirical perspective and analysis

Yuan, Hui; Tang, Yuanyuan; Xu, Wei; Lau, Raymond Yiu Keung
Internet Research , Volume 31 (3): 21 – May 19, 2021
Download PDF
21 pages

Loading next page...
 
/lp/emerald-publishing/exploring-the-influence-of-multimodal-social-media-data-on-stock-8ACOBRIhZf
Publisher
Emerald Publishing
Copyright
© Emerald Publishing Limited
ISSN
1066-2243
DOI
10.1108/intr-11-2019-0461
Publisher site
See Article on Publisher Site

Abstract

Despite the extensive academic interest in social media sentiment for financial fields, multimodal data in the stock market has been neglected. The purpose of this paper is to explore the influence of multimodal social media data on stock performance, and investigate the underlying mechanism of two forms of social media data, i.e. text and pictures.Design/methodology/approachThis research employs panel vector autoregressive models to quantify the effect of the sentiment derived from two modalities in social media, i.e. text information and picture information. Through the models, the authors examine the short-term and long-term associations between social media sentiment and stock performance, measured by three metrics. Specifically, the authors design an enhanced sentiment analysis method, integrating random walk and word embeddings through Global Vectors for Word Representation (GloVe), to construct a domain-specific lexicon and apply it to textual sentiment analysis. Secondly, the authors exploit a deep learning framework based on convolutional neural networks to analyze the sentiment in picture data.FindingsThe empirical results derived from vector autoregressive models reveal that both measures of the sentiment extracted from textual information and pictorial information in social media are significant leading indicators of stock performance. Moreover, pictorial information and textual information have similar relationships with stock performance.Originality/valueTo the best of the authors’ knowledge, this is the first study that incorporates multimodal social media data for sentiment analysis, which is valuable in understanding pictures of social media data. The study offers significant implications for researchers and practitioners. This research informs researchers on the attention of multimodal social media data. The study’s findings provide some managerial recommendations, e.g. watching not only words but also pictures in social media.

Journal

Internet Research – Emerald Publishing

Published: May 19, 2021

Keywords: Stock performance; Multimodal data; Sentiment analysis; Deep learning; Vector autoregression

References

  • Fusing text and image for event detection in Twitter
    Alqhtani, S.M.; Luo, S.; Regan, B.
  • SentiWordNet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining
    Baccianella, S.; Esuli, A.; Sebastiani, F.
  • Herd behavior in financial markets
    Bikhchandani, S.; Sharma, S.
  • Twitter mood predicts the stock market
    Bollen, J.; Mao, H.; Zeng, X.
  • Online consumer reviews: a new element of marketing communications mix
    Chen, Y.; Xie, J.
  • Wisdom of crowds: the value of stock opinions transmitted through social media
    Chen, H.; De, P.; Hu, Y.J.; Hwang, B.H.
  • Conditions for a picture-superiority effect on consumer memory
    Childers, T.L.; Houston, M.J.
  • Unit root tests for panel data
    Choi, I.
  • Trading volume and cross-autocorrelations in stock returns
    Chordia, T.; Swaminathan, B.
  • The impact of social media on c-level roles
    Deans, P.C.
  • The interaction between microblog sentiment and stock return: an empirical examination
    Deng, S.; Huang, Z.J.; Sinha, A.P.; Zhao, H.
  • Leveraging financial social media data for corporate fraud detection
    Dong, W.; Liao, S.; Zhang, Z.
  • Increasing word distinctiveness eliminates the picture superiority effect in recognition: evidence for the physical-distinctiveness account
    Ensor, T.M.; Surprenant, A.M.; Neath, I.
  • Efficient capital markets: a review of theory and empirical work
    Fama, E.F.
  • Common risk factors in the returns on stocks and bonds
    Fama, E.F.; French, K.R.
  • Emotional attention: a study of image sentiment and visual attention
    Fan, S.; Shen, Z.; Jiang, M.; Koenig, B.L.; Zhao, Q.
  • The power of prediction with social media
    Gayo-Avello, D.; Metaxas, P.T.; Mustafaraj, E.; Strohmaier, M.; Schoen, H.; Gloor, P.
  • Inducing domain to specific sentiment lexicons from unlabeled corpora
    Hamilton, W.L.; Clark, K.; Leskovec, J.
  • The time-varying nature of social media sentiments in modeling stock returns
    Ho, C.S.; Damien, P.; Gu, B.; Konana, P.
  • The puzzling media effect in the Chinese stock market
    Huang, T.
  • Testing for unit roots in heterogeneous panels
    Im, K.S.; Pesaran, M.H.; Shin, Y.
  • Image sentiment analysis using deep convolutional neural networks with domain specific fine tuning
    Jindal, S.; Singh, S.
  • Social media? Get serious! understanding the functional building blocks of social media
    Kietzmann, J.H.; Hermkens, K.; McCarthy, I.P.; Silvestre, B.S.
  • Is a picture really worth a thousand words? An experiment on hotel Facebook message effectiveness
    Leung, X.; Tanford, S.; Jiang, L.
  • News impact on stock price return via sentiment analysis
    Li, X.; Xie, H.; Chen, L.; Wang, L.; Deng, X.
  • Analyzing personality through social media profile picture choice
    Liu, L.; Preotiuc-Pietro, D.; Samani, Z.; Moghaddam, M.E.; Ungar, L.
  • Visual listening in: extracting brand image portrayed on social media
    Liu, L.; Dzyabura, D.; Mizik, N.
  • Consumer negative voice and firm-idiosyncratic stock returns
    Luo, X.
  • Customer satisfaction, analyst stock recommendations, and firm value
    Luo, X.; Homburg, C.; Wieseke, J.
  • Social media and firm equity value
    Luo, X.; Zhang, J.; Duan, W.
  • Stock prediction using event to based sentiment analysis
    Makrehchi, M.; Shah, S.; Liao, W.
  • Sentiment analysis on social media for stock movement prediction
    Nguyen, T.H.; Shirai, K.; Velcin, J.
  • Investigating predictive power of stock micro blog sentiment in forecasting future stock price directional movement
    Oh, C.; Sheng, O.R.L.
  • Glove: global vectors for word representation
    Pennington, J.; Socher, R.; Manning, C.D.
  • A systematic literature review on opinion types and sentiment analysis techniques
    Qazi, A.; Raj, R.G.; Hardaker, G.; Standing, C.
  • One picture is worth a thousand words? Not necessarily!
    Reinert, H.
  • Herd behavior and investment
    Scharfstein, D.S.; Stein, J.C.
  • Identifying and verifying news through social media: developing a user-centred tool for professional journalists
    Schifferes, S.; Newman, N.; Thurman, N.; Corney, D.; Goker, A.; Martin, C.
  • Investor sentiment and stock returns: some international evidence
    Schmeling, M.
  • Enhancing social media analysis with visual data analytics: a deep learning approach
    Shin, D.; He, S.; Lee, G.M.; Whinston, A.B.; Cetintas, S.; Lee, K.C.
  • A survey of multimodal sentiment analysis
    Soleymani, M.; Garcia, D.; Jou, B.; Schuller, B.; Chang, S.F.; Pantic, M.
  • Trading on twitter: the financial information content of emotion in social media
    Sul, H.; Dennis, A.R.; Yuan, L.I.
  • Trading on twitter: using social media sentiment to predict stock returns
    Sul, H.K.; Dennis, A.R.; Yuan, L.
  • A longitudinal study of herd behavior in the adoption and continued use of technology
    Sun, H.
  • Giving content to investor sentiment: the role of media in the stock market
    Tetlock, P.C.
  • Does chatter really matter? dynamics of user-generated content and stock performance
    Tirunillai, S.; Tellis, G.J.
  • Cross-media learning for image sentiment analysis in the wild
    Vadicamo, L.; Carrara, F.; Cimino, A.; Cresci, S.; Dell'Orletta, F.; Falchi, F.; Tesconi, M.
  • Cutting through content clutter: how speech and image acts drive consumer sharing of social media brand messages
    Villarroel Ordenes, F.; Grewal, D.; Ludwig, S.; Ruyter, K.D.; Mahr, D.; Wetzels, M.
  • The properties of retrieval cues constrain the picture superiority effect
    Weldon, M.S.; Roediger, H.L.; Challis, B.H.
  • The development of the picture-superiority effect
    Whitehouse, A.J.; Maybery, M.T.; Durkin, K.
  • A picture is not always worth a thousand words: pictures as distractors in reading
    Willows, D.M.
  • A survey on visual analytics of social media data
    Wu, Y.; Cao, N.; Gotz, D.; Tan, Y.P.; Keim, D.A.
  • Will video be the next generation of e-commerce product reviews? Presentation format and the role of product type
    Xu, P.; Chen, L.; Santhanam, R.
  • Twitter as customer's eWOM: an empirical study on their impact on firm financial performance
    Xun, J.; Guo, B.
  • An empirical examination of online restaurant reviews on Yelp.com: a dual coding theory perspective
    Yang, S.B.; Hlee, S.; Lee, J.; Koo, C.
  • Weakly supervised coupled networks for visual sentiment analysis
    Yang, J.; She, D.; Lai, Y.; Rosin, P.L.; Yang, M.
  • Robust image sentiment analysis using progressively trained and domain transferred deep networks
    You, Q.; Luo, J.; Jin, H.; Yang, J.
  • Visual sentiment analysis by attending on local image regions
    You, Q.; Jin, H.; Luo, J.