Performance Evaluation Of Classical Algorithms On Sentiment Analysis

The sentiment label of twitter data may be positive, negative, neutral etc. We try to compare the performance of those algorithms based on different real time dataset which are collected by using twitter APK. For performing performance evaluation task we build Confusion Matrix and Based on Accuracy, Precision and Recall we compare those Algorithms performance. Finally, we try to find the best algorithm among them.

I. Introduction

Sentiment Analysis is that the method of finding the opinion of user concerning some topic or the text in thought. it's conjointly called opinion mining. In alternative words, it determines whether or not a chunk of writing is positive, negative or neutral. Social media has step by step become the most supply of assorted info concerning people’s existence. this can be due to the character of social media, particularly Twitter, on which individuals post in-the-moment tweets concerning their opinions on a range of topics, discuss current social problems, complain and specific positive sentiment for merchandise they use a day. Therefore, there's a growing would like for firms World Health Organization manufacture merchandise to fetch the Twitter knowledge and obtain a way of general sentiment for his or her product.

Generally, the businesses even reply to users on their public social media. Overall, the main goal is to build a technology to detect and summarize an overall sentiment. There are several challenges to applying sentiment analysis in the business world. Firstly, how to fetch the real-time Twitter data with keywords. Usually, when events break out or new products come out, companies would like to fetch the Twitter data as early as possible, so there is a need to set an operator to fetch the data in real-time. Secondly, the main a part of the analysis method is to use sentiment analysis into the info. There are some common challenges of sentiment analysis similar to emoji analysis, word order, writing system and sure words which will have a distinct or opposite that means in sure things. Thirdly, building an interactive application for users or analysts to set the keywords they want and visualize the frequent words with word cloud graphs or word count tables. when sentiment analysis, all Twitter knowledge square measure scored and characterized into positive, neutral or negative. It seems very difficult to find the exact reasons behind sentiment variations as number of tweets are more than thousands for the target event.

Different Classes of Sentiment Analysis

There are three classes of sentiments. i. e. positive, negative and neutral sentiments. i. Positive Sentiments: This refers to positive attitude of the speaker about the text. Emotions with positive sentiments reflect happiness, joy, smile etc. In case of political reviews, if the positive reviews/sentiments about the politician are more, it means people are happy with his work. ii. ii. Negative Sentiments: This refers to negative attitude of the speaker about the text. Emotions with negative sentiments reflect sadness, jealousy, hate etc. In case of political reviews, if the negative reviews/sentiments about the politician are more, it means people are not happy with his work. iii. iii. Neutral Sentiments: Here no emotions are reflected about the text. It is neither preferred nor neglected. Although this class doesn’t imply anything, it is very important for better distinction between positive and negative classes.

II. Architectural diagram

III. Literature survey

Recently, there have been many studies about sentiment analysis for various purposes. In paper [1] by proposing an approach to automatically detect sentiment on Twitter messages (Tweets) and also propose two step sentiment analysis classification method for Twitter. First classifies messages as subjective and objective and further distinguishes the subjective tweets as positive or negative. For creating these classifiers, instead of using manually annotated data to compose the training data as regular supervised approaches, they leverage sources of noisy labels as their training data. These noisy labels were provided by a few sentiment detection websites over twitter data. For better utilizing these sources it is important to verify the potential value of using and combining them.

Providing an analysis of the provided labels and examining different strategies of combining these source in order to find the best result. A more robust feature set that captures a more abstract representation of tweets and it is composed by meta-information associated to words and specific characteristics of how tweets are written is also proposed by the authors. The contributions of paper [2] are to introduction POS-specific prior polarity features and to explore the use of a tree kernel to obviate the need for tedious feature engineering. Looking at popular micro blog Twitter, here the authors build models for two classifying tasks. These are a binary task of classifying sentiment into positive, negative classes and 3 way tasks means to classifying sentiment into positive, negative and neutral classes.

They also experiment with unigram model, a feature based model and a tree kernel based model. In the tree kernel based model they design a new tree representation for tweets. Their tree kernel based model out performs both these models by a significant margin. In the unigram based model, their experiment shows that a hard baseline is indeed for achieving over 20% over the chance baseline for both classification tasks. Their feature based model that uses only 100 features achieves similar accuracy as the unigram model that uses over 10,000 feature. By investigating the utility of linguistic features for detecting the sentiment of Twitter messages, the author of this paper[3] evaluate the usefulness of existing lexical resources as well as features to capture information about the informal and creative language used in microblogging. For building training data they take a supervised approach but leverage existing hashtags in Twitter data. In their experiment they use three different corpora of Twitter messages. For development and training they use hash tagged dataset (HASH) and the emoticon dataset (EMOT). There are three steps for preprocessing the dataset. They are

1) tokenization,

2) normalization and

3) parts of-speech (POS) tagging and they also use a variety of features for their classification and experiment. They use unigrams and bigrams for the baseline and they also include features typically used in sentiment analysis such as sentiment lexicon and POS feature.

IV. Sentiment analysis with multiple machine learning algorithms

Twitter data has an impressive predictive power. It is such a platform where people express and discuss their opinions on current issues. For analyzing the sentiment over twitter data, some machine learning algorithms are used. They are Support Vector Machines (SVM), K-Nearest Neighbors(KNN), Naïve Bayes and Maximum entropy. The basic information of such topics is given below:a) SVM (support vector Machine):For classification and regression problems, Support Vector Machines are supervised learning models. They can solve linear and non-linear problems. In classification problems SVMs are most commonly used and SVMs are based on the idea of finding a hyper plane that best divides a dataset into two classes. The goal of the line is to maximizing the margin between the points on either side of the so called decision line.

The benefit of this process is, that after the separation, the model can easily guess the target classes (labels) for new cases. It is a supervised learning algorithm. Here the result of new data is classified based on majority of k-nearest neighbor category. The basic of this algorithm is to classify a new object based on attributes and training samples. The classifiers do not use any model to fit and only based on memory. If there is a query point, then we have to find the k number of training points or objects closest to the query point. The classification is using the majority vote among the classification of the k objects. K nearest neighbor algorithm used neighborhood classification as the prediction value of the new query instance.

Naive-Bayes Classification Algorithm: The Bayesian Classification represents a supervised learning method as well as a statistical method for classification. Working on a data set with millions of records with some attributes, it is suggested to try Naive Bayes approach. Naive Bayes classifier gives great results when we use it for textual data analysis. Such as Sentiment analysis on twitter data. For understanding the main idea of Naïve Bayes Classification, we can consider the following figure.

If there are two objects that are indicated as green and red. Then we try to classify new objects to decide which class they belong. Here the algorithm is used As indicated, the objects can be classified as either GREEN or RED. Our task is to classify new cases as they arrive, i. e. , decide to which class label they belong, based on the currently exiting objects. Since there are twice as many GREEN objects as RED, it is reasonable to believe that a new case (which hasn't been observed yet) is twice as likely to have membership GREEN rather than RED. In the Bayesian analysis, this belief is known as the prior probability.

V. Working procedures

1. Extract the coaching dataset
• The downloaded polarity twitter dataset contains distinct sets of knowledge of positive and negative reviews. In preparation for training, we tend to produce 2 lists. One to store the particular review texts and another to easily label every picture show review document 'positive' or 'negative' reckoning on the set.
2. Vectorize

Convert these raw text review documents into tf-idf (Term Frequency - Inverse Document Frequency) matrix of options. The vectorizer method has parameters for deciding the brink at that to ignore frequent document and vocabulary terms.

3. Classification and testing using SVM
• The initial approach are using SVM.we tend to classify the training information with the polarity labels (positive and negative labels). This generates a model, tends to use for following step, testing.
4. 4. Extract the testing dataset
• Import polarity dataset within the same manner and tends imported the training dataset in the 1st step on top of. Then performs a prediction of the vectorized check dataset supported the generated model from the training dataset. To get accuracy score, the output of the prediction is checked against the polarity labels of the test dataset. A float number between zero and one is generated with higher accuracy values nearer to 1, lower accuracy values towards zero.
5. Classification and testing using Naïve Bayes algorithm
• A similar process to SVM is involved the classifier here is multinomial, which is better suited for discrete features and works with tf-idf matrices that created in step two. Predicting and calculating the accuracy score is obtained as in the previous step.
6. Classification and testing using KNN algorithm
• K-nearest neighbor is an approach for classification that finds a group of K objects in the training documents that are close to the test value. To classify an unlabeled object, the distance between this object and labelled object is computed and its K nearest neighbors are identified. Classification accuracy mainly depends on the chosen value of K and will be better than that of using the nearest neighbor classifier. For large data sets, K can be larger to reduce the error. A variety of features are used classification experiments.