A Fuzzy Logic in Election Sentiment Analysis: Comparison Between Fuzzy Naïve Bayes and Fuzzy Sentiment using CNN

ABSTRACT


A. INTRODUCTION
Nowadays, the application of social media are widely used by the people to express their ideas on recent event that occur in their society (Fitri et al., 2019). Twitter is one of the popular social media application which widely used as platform for their user to post a message known as tweet to the maximum limit in 280 character (Ansari et al., 2020). Indonesia is a country with one of the most active Twitter user (Alita et al., 2019). In 2020, the Regional Head Election or PILKADA is one of the topics which pique people interest in Indonesia. This PILKADA is held simultaneously in December 9 th 2020. The involvement of the current Indonesia president's relatives became one of the main attraction for the people in Indonesia. The son of the current president, Gibran Rakabuming declared himself as the candidate of the head regional of Solo together with one of the nominee from PDIP, Teguh Prakosa. Meanwhile, in PILKADA Medan, Muhammad Bobby Afif Nasution, the son in law of the president of Indonesia, also declared to become one of the nominee for the regional head of Medan election. Just like the other candidates, they also uses social media to maximize their campaign during election period which accidentally held during COVID pandemic in Indonesia. Twitter become one of the main social media which applied by the election nominee for their campaign (Bansal & Srivastava, 2018). During this campaign period, the opinion and the message from the candidates and the voter which delivered on Twitter resulting in certain unstructured data.
Sentiment analysis is one of the branches of the text mining process with the purpose to clarify the review data into certain classification (Lutfi et al., 2018). Furthermore, this sentiment analysis is applied to identify like, dislike, comment, opinion, or feedback on certain content which will be categorized into positive, negative, or neutral (Budiharto & Meiliana, 2018). There are a lot of study regarding the application of sentiment analysis which applied on different topics, such as, social media comments, products, politics, etc. (Purnomoputra & Wisesty, 2019). The sentiment analysis which specified on election topic known as Election Sentiment Analysis. For further specification from the sentiment value, the application of fuzzy logic is introduced. The sentiment analysis with further application of fuzzy logic will provide a clearer views to more accurate sentiment value. This matter will support the decision maker to make an effective decision suitable for their products or services (Rahman, 2014).
The model of the fuzzy logic is based on the fuzzy set theorem proposed by Zadeh (Zadeh, 1988). Set of fuzzy is defined as a set with the membership value ranged between "0" to "1" which describe the membership degree of certain object. If the binary logic is defined as "Yes" and "No", the data valued close to 0 is considered as "No" decision and vice versa. In sentiment analysis, the binary logic is defined as "Positive" and "Negative". In this research, "0" is associated with negative sentiment, "1" is associated with positive sentiment and the value between "0" and "1" is considered as the sentiment degree of the opinion. Moreover, the neutral sentiment is also considered, with the value of "0.5". In sentiment analysis, the application of fuzzy logic is used in three different methods. The first method is by applying fuzzy logic directly on sentiment analysis (Rahman, 2014) (Jefferson et al., 2017). The second one is by combining fuzzy method with one of traditional sentiment analysis which is the Fuzzy Naïve Bayes (Fathurochman et al., 2104). Then, the last one is by applying fuzzy logic together with machine learning Convolutional Neural Networks (Fathurochman et al., 2104) (Fathurochman et al., 2104) (Fathurochman et al., 2104).
The Fuzzy Naïve Bayes is a modification method which combine two methods, which are fuzzy logic and Naïve Bayes with the objective to provide better output. Fuzzy logic is a method to solve a problem which have certain level of ambiguity. In the system which needs complexity, fuzzy logic is difficult to apply and tends to provide long time for define the suitable rules and membership settings. Meanwhile, Naïve Bayes Classifier is a statistic classification method which have the ability to predict the probability of certain classification with the application of Bayes theorem. The theorem of Bayes is a theory which used to measure the class probability from each of the variables classification. Naïve Bayes explains simple decision, which is the relation between each variable in certain classification is fully independent with another variables. This simple assumption creates the prediction which is not quite optimum, and leads to inaccuracy of the output, especially for the domain text problems (Fathurochman et al., 2104). Tree and K-NN also state that accuracy and speed are the most supportive factors in data classification (Fathurochman et al., 2104). By combining these two methods, hopefully the application of Fuzzy Naïve Bayes will solve the ambiguity problem on forecasting the certain class membership matter. Other than the traditional method such as Naïve Bayes, another different method of sentiment analysis which applied in this research is model machine learning.
Sentiment analysis can be applied based on dictionary or machine learning (Fathurochman et al., 2104). Machine learning works based on the algorithm in the data classification which labelled manually. The data which trained by machine learning gives an impactful effect on the resulted classification model, the higher the value the better the output and vice versa. Hence, sentiment analysis based on machine learning will provides a dynamic advantage to meet the new dataset languages or words. The approach by applying this method is better than the approach based on dictionary. One of the popular algorithm in machine learning is convolutional neural networks.
Based on the previous research, convolutional neural networks is proven to be one of the branches of deep learning which provides better level of accuracy (Fathurochman et al., 2104). However, the output of the sentiment analysis is a clear classification, which are positive, negative, and neutral (Sánchez-Rada & Iglesias, 2019). In CNN model, the classification result is acquired by selecting the highest probability value on each of the classification output. Another form of the sentiment analysis is sentiment score, where the output of this method is valued as positive, negative, or zero. However, this sentiment score is resulted by the application of sentiment analysis based on dictionary. In fact, the sentiment value on every sentences is not fully positive, negative, or neutral, there will always be ambiguity level there, and the concept of fuzzy logic is one of the good method to define that ambiguity.
This research is done based on the many research of sentiment analysis method application in a lot of problems in real life (Fitri et al., 2019), (Alita et al., 2019), (Susanti et al., 2017) (Andrianti, 2018) (Alamsyah, 2018) (Deriu & Cieliebak, 2016) (Nugroho et al., 2015), (Lutfi et al., 2018) (Dey et al., 2016). Based on the statement above, the fuzzy naïve Bayes sentiment analysis and fuzzy sentiment analysis with convolutional networks are chosen, applied, and compared with the objective to analyse the role of fuzzy logic in traditional sentiment method and machine learning.

B. METHODS
This research focuses on the result comparison of the sentiment analysis by Fuzzy Naïve Bayes method and Fuzzy Sentiment method on PILKADA 2020 in Solo and Medan:

Fuzzy Naïve Bayes
Naive Bayes Classifier Naïve Bayes Classifier is a machine learning technique based on the theory of probability. Although this method is simple, but this method is proven to provides high level of accuracy and performance in text classification. There are two main processes in Naïve Bayes Classifier method, which are training process and testing process. The training process will result in the rules of classification which contain the probability value for each variable (conditional probability) and the targeted classification from the data train that already determined. The testing process is a testing process for each variables to determine the optimum class based on the classification rules which already acquired (Fathurochman et al., 2104).
In Naïve Bayes Classifier, every "tweet" data are defined as the combination of certain variable ( 1 , 2 , 3 , … , ) where x1 is the first word, x2 is the second word, and so on. Then, there are V value which defined as the set of classification. After the pre-processing step already done, the classification process in Naïve Bayes Classifier is applied to clarify the data before further step is initiated. There are 4 processes which applied on classification process, such as: a. Establishment of trained data variable This process is an initial step in Naïve Bayes Classifier classification process, where the mentioned variable is described as tweet (comments) from Twitter which will be classified into sentiment class; positive, negative, or neutral by observing the emerged frequency on every words in the variable. In classification process, this method will provide a class with the highest probability value (VMAP) from the tested data in each class for every variable combination ( 1 , 2 , 3 , … , ). The value of maximum probability (VMAP) is determined by using the following equation: Fuzzy Association If X is a set of object which generally notated as x, then the set of fuzzy ̃ in X is the row combination of: ̃( ) is defined as the membership function of x in set of M between 0 to 1 (Honda & Ohsato, 1986).

Fuzzy Membership Function
The membership function is defined as a curve which shows the pattern of every input data coordinates into its membership value (membership degree) which ranged from 0 to 1 interval (Honda & Ohsato, 1986). One of the function that can be applied is the triangle curve.
The triangle curve is a combination of two linear lines as described in Figure 1.
Confusion Matrix Matrix confusion can be formed by using this following equation: Based on that equation, accuracy level by applying 2 or more n class is (Manliguez, 2016): (4)

Fuzzy Sentiment with Convolution Neural Networks
Fuzzy sentiment method by convolutional neural network tries to create sentiment value in the form fuzzy, which is between 0 and 1. This method is included as one of the part in machine learning process. The initial steps to apply in this method are data cleaning, tokenization, encoding, and padding. This process is described in figure 2. Data cleaning is a process where the unwanted data from the words or characters which being stated in the opinion are cleaned. This purpose of this process is to diminish the data which will affect the output of sentiment analysis process. The next step is tokenization, this step means to break the sentences into single words. Then, this single words will be encoded into numbers to be structured into sentence vector. The length difference in each sentences then will be uniformed by applying padding method. After the padding process is finalized, the measurement of the sentiment is applied by using Convolutional Neural Network method. The structure of convolutional neural networks is consisted into 3 main layers, which are input layer, feature extraction layer, and classification layer (Nasichuddin et al., 2018). This structure is described in figure 3. For text classification, embedding process is applied in input layer. The words insertion is a process to interpret the words as a bulk vector. This method is applied to identify the adjacency between words through vector representation. This process will be described thoroughly in figure 4. In the next layer, convolution process will be applied. The term of convolution refers to mathematics combination from 2 function to form the next third function. The objective of this process is for feature extraction so that the only features which will be considered in neural networks model are the important one. To prevent overfitting, dropout process is applied. Dropout is a technique where the unnecessary neurons which are randomly chosen will be ignored during training (Melucci, 2016). This means that their contribution towards downstream neuron activation is eliminated temporarily in forward pass process and every weight updating process won't be affected by neuron which applied in backward pass. Dropout structure will be shown in figure 5. Maxpooling process then will be applied to reduce the computational weight of this algorithm. For each classification, the suitable activation function is applied. For binary classification, sigmoid activation function will be used while softmax activation function is applied for multiclass classification. The result of this modelling process is probability value on each class classification. Class featured with the highest value of probability will be considered as the referred object. Then, these result will be modified into set of fuzzy. The modification of model result into set of fuzzy will be mentioned on figure 6(a) for binary class and figure 6(b) for multiclass. The output of this method are vector of sentences sentiment value on dataset. The similarity in this vector will be measured by applying Euclid distance concept with dataset label sentiment vector. The Euclid distance between two vectors, ⃗ = ( 1 , 2 , ⋯ , ) and ⃗ = ( 1 , 2 , ⋯ , ) inside Euclidean space is defined as follows (Strang, 2016): The distance between fuzzy sentiment vector and standard convolutional neural networks sentiment vector will be compared. The distance result which is lower means that this vector is close to the dataset label sentiment vector. This concludes that the applied sentiment value is better, or close to the real output.

Dataset
Dataset is defined as the tweets which are taken from twitter starting from 26 September 2020 to 6 December 2020. In PILKADA Solo, the two candidates who compete are Gibran Rakabuming -Teguh Prakosa as candidate one, and Bagyo Wahyono -FX Supardjo as candidate two (KPUD Kota Surakarta, 2020). The applied keywords for this parameter are Gibran-Teguh, and Bagyo-Supardjo. In addition to that, in PILKADA Medan(KPUD Kota Medan, 2020), the competing candidates are Akhyar Nasution -Salman Al-Farisi as candidate one, and Muhammad Bobby Afif Nasution -Aulia Rahman as candidate two. The applied keywords for this parameter are Akhyar-Salman, and Bobby-Aulia. The number of tweets which collected are described in Table 1.

C. RESULT AND DISCUSSION
Fuzzy Naïve Bayes method resulting on the percentage value of sentiment for each candidate. This percentage is illustrated in figure 7. Based on the output data in figure 7, candidate Gibran-Teguh receive 27.915% positive sentiment, 55.008% negative sentiment, and 17.077% neutral sentiment with level of accuracy valued 66.9%. Meanwhile, for candidate Bagyo-Supardjo, resulting in 33.333%, 54.167%, and 12.500% for positive sentiment, negative sentiment, and neutral sentiment respectively with accuracy percentage of 54.16%. Furthermore, for PILKADA Medan, candidate Akhyar-Salman received 21.577% positive sentiment, 63.900% negative sentiment, and 14.523% neutral sentiment in 63.48% accuracy. For candidate 2, Bobby-Aulia received 26.718%, 50,725%, and 17.557% for positive, negative, and neutral sentiment respectively in 67.17% accuracy. These data mentioned before shows that the highest sentiment percentages received for four nominees is the negative ones. Next, the average result for the sentiment fuzzy with convolutional neural network is illustrated on table 2. While for the trend of sentiment value for each week is shown on figure 8. Based on these data, the sentiment value for candidates in PILKADA Solo tends to be positive with the acquired value higher than 0.5. Meanwhile, the sentiment value for candidates in PILKADA Medan tends to be negative with the acquired value lower than 0.5. Nevertheless, the last acquired result actually linear with the Fuzzy Naïve Bayes method. Besides that, the transformation of the sentiment for each week in campaign period is also fluctuates.  On this comparison, for the candidate in PILKADA Medan, the one who receive higher positive sentiment percentage in Fuzzy Sentiment CNN and Fuzzy Naïve Bayes method successfully win the election. Meanwhile, the opposite comparison achieved for PILKADA Solo, the candidate who receive lower positive sentiment percentage in Fuzzy Sentiment CNN and Fuzzy Naïve Bayes method, in fact acquiring higher number of votes and win in a huge margin. This phenomenon occur because the twitter data for the losing candidate which are collected for this research are too low, only 119 tweets. This conclusion concludes by considering the concept of Machine Learning where the sentiment analysis process will works better in bigger dataset.

D. CONCLUSION AND SUGGESTIONS
This research concludes that the two applied method are well implemented. The concept of fuzzy logic have the ability to solve the ambiguity problem in sentiment analysis. By applying huge number of data, Fuzzy Naïve Bayes and Fuzzy Sentiment with Convolutional Neural Network could generate the sentiment value which represent the real count result in PILKADA. In the future, the application of fuzzy logic should be perfected in sentiment analysis method. This statement is important considering that ambiguity is the main and general problem to determine the sentiment from certain statement.