Sentiment Analysis is the process of determining whether the text piece expresses a positive, negative or neutral sentiment. It is also known as opinion mining, deriving the attitude of a writer. It is widely used to determine the expression of the people such as reviews and survey responses, online and social media posts for applications that range from marketing to customer support.

Its common use case is to determine how people are thinking about a particular topic, for example, you want to know how much a political leader is famous and what are the public sentiments towards him. This is generally done to predict the result of an election. Analyzing the public sentiments by their recent tweets towards a candidate, we can have an estimate of his chances of winning the election. This is exactly what was implemented during 2016 election in the US to predict their public sentiments towards Donald Trump and Hilary Clinton.

Sentiment Analysis is done by determining the polarity of the given sentence, document, etc. Polarity classifies the text and analyzes the emotion behind each word; whether the word reflects the emotion of hatred, love, anger, etc. Followed by polarity classification, the polarity of the whole document is determined by its polarity score.

I have done a use case for sentiment analysis on twitter feed i.e., tweets. This use case fetches all the tweets in real-time from twitter stream and filters it accordingly, as per our need. A dashboard shown below {Fig. (a)} was designed to show sentiments of people towards 3 major political parties of India in the recent elections. Needless to say that sentiment analysis is also based on the emoticons used in the tweet, the same provides more clarification and classification of the tweet.

The tweets are filtered based on the parties, people have mentioned, followed by their opinions about those political parties. This gives us a quick view and rough idea of what people are thinking about the leaders.

Sentiment Analysis Workflow:

Text Input > Tokenization > Stop word Filtering > Polarity Scoring > Classification

Fig. (a) Dashboard showing Sentiment Analysis of Twitter Feeds

The Geek Side of Sentiment Analysis:

The technologies used here are Elasticsearch and Kibana which was discussed in my previous blog.

A powerful, flexible, real-time and distributed. The high stream of real-time data is managed by AWS services which include AWS Kinesis Streams, AWS S3 for storing the data and AWS Elasticsearch cluster for searching and presenting the data in a graphical manner.

The analytics part is done by a python script which gives the polarity score to each tweet by analyzing the text and emoticons. My implementation of Sentiment Analysis uses python library TextBlob for classifying the text according to sentiment. This gives the polarity score for each tweet.

In the above screenshot, you can clearly see the classification of sentiments into neutral, positive and negative, further classified into political parties. This gives an idea about the popularity of a party and helps in understanding the public opinion about them.

Kibana has the power to display the real-time result as it is backed up by powerful Elasticsearch engine and also has a feature to drill down to any granular level you want, say if you want to see the text which reflects negative sentiment for a particular party then, you can do that with a single click.

The whole system works in real-time and allows you to see the analytics data of a certain period given that you are storing the data. I used AWS S3 for that part.

That’s all about the geek part. If you have any question, feel free to comment down.

Why Sentiment Analysis?

This should not be a question as of now since we have been discussing what real-time analytics is capable of, all the way through. Data is growing rapidly, and analyzing it gets you useful information for your business, customer service, customer experience and how your product is performing. Sentiment Analysis does the same; how much sales your product is acquiring, what your customers are thinking about your product or your service, and so on. This can be done not only on twitter data or any social media data but also on your customer feedback.

Hope this excerpt was of some use to you. Keep enhancing your product or service to improve customer experience and engagement till I come up with some new interesting slot on Analytics.


Please enter your comment!
Please enter your name here