Custom Word Embeddings for Sentiment Analysis

Abstract

Our work analyzed the relationship between the domain type of the word embeddings used to create sentiment analysis models and the domain of the sentiment analysis task. We used 3 corpora, (NYT, LION poems, and Book blurbs) to create our custom word embeddings and developed 9 different sentiment analysis models from our best word embeddings for each of the corpora. To compare our custom word embeddings against, we used 8 other pre-trained sentiment models including XLNet’s model, BERT (6 variations), and a simple skip-gram model used with Google vectors. Though we were hoping to prove that there was a clear advantage to using similar domains for the word embeddings as the dataset your sentiment model would be tested on, we concluded that more generally trained embeddings will outperform in-domain word embeddings.