TIPS :
Home » Python Resources » A Complete Guide to Naive Bayes Algorithm in Python

# A Complete Guide to Naive Bayes Algorithm in Python

Naive Bayes is a classification algorithm for binary class and multiclass classification problems. Naive Bayes is applied on each row and column. It is used for text classification.

## What is Probability?

``The chance of something happening is called ‘probability.’``

Probability = No. of outcome / Total No. of outcome

• Flipping a Coin(H,T) = ½ — Independent Event
Rolling a Dice, P(5) = ⅙ — Independent Event
Let’s take both events
P(H,5)= P(H) * P(5) —Joint Event
• Event A = Taking a blue marble
P(B)=⅗
``````Event B= Taking Second blue marble
P(2B)=2/4 = ½ ``````

P(getting 2nd blue marble / when already 1st blue marble has been taken out)=½

Event A and B are dependent events, hence we will use Conditional Probability.

## General Formula

Let’s look at an example :
Task: Based on weather conditions predict if the player will play or not.

Step 1: Make a Frequency table

``                     FREQUENCY TABLE    ``

Step 2 : Create a Likelihood table

``                                LIKELIHOOD TABLE                ``

The problem statement is whether the players would play if the weather is sunny.

P(Yes/Sunny) = P(Sunny/Yes) * P(Yes) / P(Sunny)
= 3/9 * 9/14 / 5/14
P ( No / Overcast ) = P ( Overcast / No ) * P ( No ) / P (Overcast)
=0/5 * 5/14 / 0/14
= 0

Naive Bayes Algorithm Explained

## Feature Vector Creation

The feature vector is the method which converts text data into feature vectors. Feature vector creation can be done by
Bag of words

TFIDF

Bag of words

Step1: Tokenization ( Removing Stopwords)

``````                                                                       TOKENIZATION
Sentence 1: He is a good boy.                  — good boy
Sentence 2: She is a good girl.                 — good girl
Sentence 3: Both are good boy and girl    — good boy girl``````

Step2: Feature vector creation ( unique tokens are getting features )

``````f1    f2      f3
Good     boy    girl``````

Sent1 1 1 0

Sent2 1 0 1

Sent3 1 1 1

## TF-IDF

TF-IDF is short for Term Frequency Inverse Document Frequency

``````TF = ( No. of times the word repeated in sentence /
Total No. of words in a sentence)

IDF = log ( Total No. of sentence /
No.of sentences containing that word )``````

Step1: Tokenization ( Removing Stopwords )

``````         Sentence 1:    — good boy
Sentence 2:    — good girl
Sentence 3:    — good boy good girl``````

Step2: Creating TF table

``````        Sent1   Sent2   Sent3

Good         1/2             1/2               1/3

Boy            1/2               0                1/3

Girl              0               1/2               1/3``````

Step3: Creating IDF table

Good log( 3 / 3 ) = log(1) =0

Boy log ( 3 / 2 )

Girl log ( 3 / 2 )

Step4: Feature vector creation

``      Good            Boy               Girl``

Sent1 1/2 * 0 1/2 * log( 3 / 2 ) 0

Sent2 1/2 * 0 0 1/2 * log( 3 / 2 )

Sent3 1/3 * 0 1/3 * log( 3 / 2 ) 1/3 * log( 3 / 2 )

## 3 Types of Naive Bayes in Scikit Learn

### Gaussian

• It is used for classification problems and it assumes that features have a normal distribution.

### Multinominal

• It is used for discrete counts.

### Bernoulli

• It is used for binary counts(ie., Zeroes and One).

## Pros of Naive Bayes

• Naive Bayes is a fast and highly scalable algorithm.
• Naive Bayes can be classified into both binary classification and multi-class classification. It has different types of Naive Bayes Algorithms like GaussianNB, MultinominalNB, and BernoulliNB.
• The algorithm is simple and depends on doing a bunch of counts.
• Best choice for text classification problems. Widely used in spam mail classification.
• Training on small datasets is easy.

## Cons of Naive Bayes

• It cannot learn the relationship between features because it considers all the features to be unrelated.

## Application of Naive Bayes

• Naive Bayes is broadly utilized for text classification.
• News article classification SPORTS, TECHNOLOGY etc.
• Spam or Ham: Naive Bayes is the most popularly used for mail filtering.

## Python Implementation of Naive Bayes

• Importing the required libraries
• import pandas as pd
• from sklearn.model_selection import train_test_split
• from sklearn.feature_extraction.text import CountVectorizer,TfidfVectorizer
• from sklearn.naive_bayes import MultinomialNB
• from sklearn.metrics import accuracy_score
• import string
• import matplotlib.pyplot as plt

Refer to the following articles:

Why is Python an Interpreted Programming Language?

Scala vs Python for Apache Spark

What is Python Programming?

data.head(8) # View the first 8 records of our dataset

Output:

The mails are categorized into 2 classes ie., spam and ham.

#Let’s see the count of each class

data.groupby(‘Class’).count()

Output:

Text PreProcessing

#Lets assign ham as 1

data.loc[data[‘Class’]==”ham”,”Class”] = 1

#Lets assign spam as 0

data.loc[data[‘Class’]==”spam”,”Class”] = 0

Removing punctuations

#the default list of punctuations

import string
String.punctuation

Output:
‘!”#\$%&\'()*+,-./:;<=>?@[\]^_`{|}~’

#Let’s remove the punctuation

def remove_punct(text1):
text1 = “”.join([c for c in text if c not in string.punctuation])
return text1
data[‘text_clean’] = data[‘Message’].apply(lambda x: remove_punct(x))

Output:

#Tokenization

#Countvectorizer is used to convert text to numerical data.

#Initialize the object for countvectorizer

CV = CountVectorizer(stop_words=”english”)

Splitting x and y
xSet = data[‘text_clean’].values
ySet = data[‘Class’].values
ySet

Splitting Train and Test Data
xSet_train,xSet_test,ySet_train,ySet_test = train_test_split(xSet,ySet,test_size=0.2, random_state=10)

xSet_train_CV = CV.fit_transform(xSet_train)
xSet_train_CV

### Feature vectorizer

#text preprocessing and feature vectorizer

#To extract features from a document of words, we import TfidfVectorizer

from sklearn.feature_extraction.text import TfidfVectorizer
tf=TfidfVectorizer() ## object creation
X=tf.fit_transform(X) ## fitting and transforming the data into vectors

#Training the model

#Creating training and testing

from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(X,y,random_state=6)

#Model creation

from sklearn.naive_bayes import MultinomialNB

#model object creation

nb=MultinomialNB()

#fitting the model

nb.fit(X_train,y_train)

#Prediction

#getting the prediction

y_hat=nb.predict(X_test)
y_hat

#Evaluation

#Evaluating the model

from sklearn.metrics import classification_report,confusion_matrix
print(classification_report(y_test,y_hat))

SpamClassificationApplication
msg = input(“Enter Message: “) # to get the input message
msgInput = CV.transform([msg]) #
predict = NB.predict(msgInput)
if(predict==0):
print(“————————MESSAGE-SENT-[CHECK-SPAM-FOLDER]—————————“)
else:
print(“—————————MESSAGE-SENT-[CHECK-INBOX]——————————“)

Output:

Being a prominent data science institute, DataMites provides specialised training in topics including deep learning, machine learning, artificial intelligence, the internet of things, and Python. Our Machine Learning Courses at DataMites have been authorised by the International Association for Business Analytics Certification (IABAC), a body with a strong reputation and high appreciation in the analytics field.

XGBOOST in Python (Hyper parameter tuning)

Reinforcement Learning in Python with Simple Example DataMites Team publishes articles on Data Science, Machine Learning, and Artificial Intelligence periodically. These articles are meant for Data Science aspirants (Beginners) and for those who are experts in the field. It highlights the latest industry trends that will help keep you updated on the job opportunities, salaries and demand statistics for the professionals in the field. You can share your opinion in the comments section. Datamites Institute provides industry-oriented courses on Data Science, Artificial Intelligence and Machine Learning. Some of the courses include Python for data science, Machine learning expert, Artificial Intelligence Expert, Statistics for data science, Artificial Intelligence Engineer, Data Mining, Deep Learning, Tableau Foundation, Time Series Foundation, Model deployment (Flask-API) etc.
x

## What is the Salary for Python Developer in India?

Python is leading the way in programming, which is the future of the planet. Its popularity is increasing tremendously with each passing year. Python is ...

## Is Data Science & Artificial Intelligence in Demand in South Africa?

According to the Economic Complexity Index, South Africa was the world’s number 38 economy in terms of GDP (current US\$) in 2020, number 36 in ...