Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Exploring the effectiveness of word embedding based deep learning model for improving email classification

Exploring the effectiveness of word embedding based deep learning model for improving email... Classifying emails as ham or spam based on their content is essential. Determining the semantic and syntactic meaning of words and putting them in a high-dimensional feature vector form for processing is the most difficult challenge in email categorization. The purpose of this paper is to examine the effectiveness of the pre-trained embedding model for the classification of emails using deep learning classifiers such as the long short-term memory (LSTM) model and convolutional neural network (CNN) model.Design/methodology/approachIn this paper, global vectors (GloVe) and Bidirectional Encoder Representations Transformers (BERT) pre-trained word embedding are used to identify relationships between words, which helps to classify emails into their relevant categories using machine learning and deep learning models. Two benchmark datasets, SpamAssassin and Enron, are used in the experimentation.FindingsIn the first set of experiments, machine learning classifiers, the support vector machine (SVM) model, perform better than other machine learning methodologies. The second set of experiments compares the deep learning model performance without embedding, GloVe and BERT embedding. The experiments show that GloVe embedding can be helpful for faster execution with better performance on large-sized datasets.Originality/valueThe experiment reveals that the CNN model with GloVe embedding gives slightly better accuracy than the model with BERT embedding and traditional machine learning algorithms to classify an email as ham or spam. It is concluded that the word embedding models improve email classifiers accuracy. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Data Technologies and Applications Emerald Publishing

Exploring the effectiveness of word embedding based deep learning model for improving email classification

Download PDF
23 pages

Loading next page...
 
/lp/emerald-publishing/exploring-the-effectiveness-of-word-embedding-based-deep-learning-M3jyWC9ULD
Publisher
Emerald Publishing
Copyright
© Emerald Publishing Limited
ISSN
2514-9288
DOI
10.1108/dta-07-2021-0191
Publisher site
See Article on Publisher Site

Abstract

Classifying emails as ham or spam based on their content is essential. Determining the semantic and syntactic meaning of words and putting them in a high-dimensional feature vector form for processing is the most difficult challenge in email categorization. The purpose of this paper is to examine the effectiveness of the pre-trained embedding model for the classification of emails using deep learning classifiers such as the long short-term memory (LSTM) model and convolutional neural network (CNN) model.Design/methodology/approachIn this paper, global vectors (GloVe) and Bidirectional Encoder Representations Transformers (BERT) pre-trained word embedding are used to identify relationships between words, which helps to classify emails into their relevant categories using machine learning and deep learning models. Two benchmark datasets, SpamAssassin and Enron, are used in the experimentation.FindingsIn the first set of experiments, machine learning classifiers, the support vector machine (SVM) model, perform better than other machine learning methodologies. The second set of experiments compares the deep learning model performance without embedding, GloVe and BERT embedding. The experiments show that GloVe embedding can be helpful for faster execution with better performance on large-sized datasets.Originality/valueThe experiment reveals that the CNN model with GloVe embedding gives slightly better accuracy than the model with BERT embedding and traditional machine learning algorithms to classify an email as ham or spam. It is concluded that the word embedding models improve email classifiers accuracy.

Journal

Data Technologies and Applications – Emerald Publishing

Published: Aug 23, 2022

Keywords: Email classification; Machine learning; Word embedding; GloVe; BERT; Deep learning

References

  • Classification of poetry text into the emotional states using deep learning technique
    Ahmad, S.; Asghar, M.Z.; Alotaibi, F.M.; Khan, S.
  • Using the contextual language model BERT for multi-criteria classification of scientific articles
    Ambalavanan, A.K.; Devarakonda, M.V.
  • Deep neural network and model-based clustering technique for forensic electronic mail author attribution
    Apoorva, K.A.; Sangeetha, S.
  • Benchmarking performance of machine and deep learning-based methodologies for Urdu text document classification
    Asim, M.N.; Ghani, M.U.; Ibrahim, M.A.; Mahmood, W.; Dengel, A.; Ahmed, S.
  • Malicious text identification: deep learning from public comments and emails
    Baccouche, A.; Ahmed, S.; Sierra-Sosa, D.; Elmaghraby, A.
  • Email spam classification using gated recurrent unit and long short-term memory
    Basyar, I.; Adiwijaya; Murdiansyah, D.T.
  • Authorship identification on limited samplings
    Boran, T.; Martinaj, M.; Hossain, M.S.
  • E-mail classification with machine learning and word embeddings for improved customer support
    Borg, A.; Boldt, M.; Rosander, O.; Ahlstrand, J.
  • A hybrid BERT model that incorporates label semantics via adjustive attention for multi-label text classification
    Cai, L.; Song, Y.; Liu, T.; Zhang, K.
  • Learning word dependencies in text by means of a deep recurrent belief network
    Chaturvedi, I.; Ong, Y.S.; Tsang, I.W.; Welsch, R.E.; Cambria, E.
  • Ensemble application of convolutional and recurrent neural networks for multi-label text categorization
    Chen, G.; Ye, D.; Xing, Z.; Chen, J.; Cambria, E.
  • Stacked DeBERT: all attention in incomplete data for text classification
    Cunha Sergio, G.; Lee, M.
  • Machine learning for email spam filtering: review, approaches and open research problems
    Dada, E.G.; Bassi, J.S.; Chiroma, H.; Abdulhamid, S.M.; Adetunmbi, A.O.; Ajibuwa, O.E.
  • Spam filtering using a logistic regression model trained by an artificial bee colony algorithm
    Dedeturk, B.K.; Akay, B.
  • An assessment of deep learning models and word embeddings for toxicity detection within online textual comments
    Dessì, D.; Recupero, D.R.; Sack, H.
  • BERT: pre-training of deep bidirectional transformers for language understanding
    Devlin, J.; Chang, M.W.; Lee, K.; Toutanova, K.
  • Using extreme gradient boosting to identify origin of replication in Saccharomyces cerevisiae via hybrid features
    Do, D.T.; Le, N.Q.K.
  • A hybrid deep learning approach for stock price prediction
    Dutta, A.; Pooja, G.; Jain, N.; Panda, R.R.; Nagwani, N.K.
  • The Enron-Spam datasets
  • Text classification based on convolutional neural networks and word embedding for low-resource languages: tigrinya
    Fesseha, A.; Xiong, S.; Emiru, E.D.; Diallo, M.; Dahou, A.
  • Machine intelligence-based algorithms for spam filtering on document labeling
    Gaurav, D.; Tiwari, S.M.; Goyal, A.; Gandhi, N.; Abraham, A.
  • A hybrid CNN-LSTM model for SMS spam detection in Arabic and English messages
    Ghourabi, A.; Mahmood, M.A.; Alzubi, Q.M.
  • Fake consumer review detection using deep neural networks integrating word embeddings and emotion mining
    Hajek, P.; Barushka, A.; Munk, M.
  • SeFACED: semantic-based forensic analysis and classification of E-mail data using deep learning
    Hina, M.; Ali, M.; Javed, A.R.; Ghabban, F.; Khan, L.A.; Jalil, Z.
  • Sentiment analysis in Bengali via transfer learning using multi-lingual BERT
    Islam, K.I.; Islam, M.S.; Amin, M.R.
  • Bi-LSTM model to increase accuracy in text classification: combining word2vec CNN and attention mechanism
    Jang, B.; Kim, M.; Harerimana, G.; Kang, S.U.; Kim, J.W.
  • The growing cyber threats for Digital India
  • Radiomics-based machine learning model for efficiently classifying transcriptome subtypes in glioblastoma patients from MRI
    Le, N.Q.K.; Hung, T.N.K.; Do, D.T.; Lam, L.H.T.; Dang, L.H.; Huynh, T.T.
  • A generative Model for category text generation
    Li, Y.; Pan, Q.; Wang, S.; Yang, T.; Cambria, E.
  • Sequence encoding incorporated CNN model for Email document sentiment classification
    Liu, S.; Lee, I.
  • Document-level multi-topic sentiment classification of Email data with BiLSTM and data augmentation
    Liu, S.; Lee, K.; Lee, I.
  • Sentic LSTM: a hybrid network for targeted aspect-based sentiment analysis
    Ma, Y.; Peng, H.; Khan, T.; Cambria, E.; Hussain, A.
  • A comparative study of deep neural network models on multi-label text classification in finance
    Maia, M.; Sales, J.E.; Freitas, A.; Handschuh, S.; Endres, M.
  • Well-calibrated confidence measures for multi-label text classification with a large number of labels
    Maltoudoglou, L.; Paisios, A.; Lenc, L.; Martínek, J.; Král, P.; Papadopoulos, H.
  • A lifelong spam emails classification model
    Mohammad, R.M.A.
  • Hierarchical and lateral multiple timescales gated recurrent units with pre-trained encoder for long text classification
    Moirangthem, D.S.; Lee, M.
  • Word-class embeddings for multiclass text classification
    Moreo, A.; Esuli, A.; Sebastiani, F.
  • Detection of spam and threads identification in E-mail spam corpus using content based text analytics method
    Murugavel, U.; Santhi, R.
  • GloVe: global vectors for word representation
    Pennington, J.; Socher, R.; Manning, C.D.
  • A novel LSTM–CNN–grid search-based deep neural network for sentiment analysis
    Priyadarshini, I.; Cotton, C.
  • Classification model of contact center customers emails using machine learning
    Putong, M.W.; Suharjito
  • Identifying SNAREs by incorporating deep learning architecture and amino acid embedding representation
    Quoc, N.; Le, K.; Huynh, T.
  • Citation intent classification using word embedding
    Roman, M.; Shahid, A.; Khan, S.; Koubaa, A.; Yu, L.
  • Exploiting contextual word embedding of authorship and title of articles for discovering citation intent classification
    Roman, M.; Shahid, A.; Uddin, M.I.; Hua, Q.; Maqsood, S.
  • Learn#: a novel incremental learning method for text classification
    Shan, G.; Xu, S.; Yang, L.; Jia, S.; Xiang, Y.
  • ML-EC2: an algorithm for multi-label email classification using clustering
    Sharaff, A.; Nagwani, N.K.
  • Index of/old/publiccorpus
  • Daily number of e-mails worldwide 2025 | Statista
  • Cognition based spam mail text analysis using combined approach of deep neural network classifier and random forest
    Sumathi, S.; Pugalendhi, G.K.
  • Sentiment analysis in textual, visual and multimodal inputs using recurrent neural networks
    Tembhurne, J.V.; Diwan, T.
  • A study of boosted evolutionary classifiers for detecting spam
    Trivedi, S.K.; Dey, S.
  • A survey of word embeddings based on deep learning
    Wang, S.; Zhou, W.; Jiang, C.
  • Deep learning algorithm for judicial judgment prediction based on BERT
    Wang, Y.; Gao, J.; Chen, J.
  • A feature-centric spam email detection model using diverse supervised machine learning algorithms
    Zamir, A.; Khan, H.U.; Mehmood, W.; Iqbal, T.; Akram, A.U.
  • Towards scalable and reliable capsule networks for challenging NLP applications
    Zhao, W.; Peng, H.; Eger, S.; Cambria, E.; Yang, M.
  • A comparative review on deep learning models for text classification
    Zulqarnain, M.; Ghazali, R.; Hassim, Y.M.M.; Rehan, M.