Citation: Proasha, N.J.; Sam, A.A.;
Kowsher, M.; Murad, S.A.; Bairagi,
A.K.; Masud, M.; Baz, M. Transfer
Learning for Sentiment Analysis
Using BERT Based Supervised
Fine-Tuning. Sensors 2022, 22, 4157.
hps://doi.org/10.3390/s22114157
Academic Editors: Yangquan Chen,
Nunzio Cennamo, M. Jamal Deen,
Subhas Mukhopadhyay, Simone
Morais, Junseop Lee and Roberto Teti
Received: 1 March 2022
Accepted: 11 May 2022
Published: 30 May 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional al-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Swierland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Aribution (CC BY) license (hps://
creativecommons.org/licenses/by/
4.0/).
Article
Transfer Learning for Sentiment Analysis Using BERT Based
Supervised Fine-Tuning
Nusrat Jahan Proasha
1
, Abdullah As Sami
2
, Md Kowsher
3,
* , Saydul Akbar Murad
4
,
Anupam Kumar Bairagi
5
, Mehedi Masud
6
and Mohammed Baz
7
1
Department of Computer Science and Engineering, Daodil International University,
Dhaka 1341, Bangladesh; jahannusratproa@gmail.com
2
Department of Computer Science & Engineering, Chiagong University of Engineering & Technology,
Chaogram 4349, Bangladesh; abdullahassami@gmail.com
3
Department of Computer Science, Stevens Institute of Technology, Hoboken, NJ 07030, USA
4
Faculty of Computing, Universiti Malaysia Pahang, Pekan 26600, Malaysia; saydulakbarmurad@gmail.com
5
Computer Science and Engineering Discipline, Khulna University, Khulna 9208, Bangladesh;
anupam@ku.ac.bd
6
Department of Computer Science, College of Computers and Information Technology, Taif University,
P.O. Box 11099, Taif 21944, Saudi Arabia; mmasud@tu.edu.sa
7
Department of Computer Engineering, College of Computers and Information Technology, Taif University,
P.O. Box 11099, Taif 21944, Saudi Arabia; mo.baz@tu.edu.sa
* Correspondence: ga.kowsher@gmail.com
Abstract:
The growth of the Internet has expanded the amount of data expressed by users across
multiple platforms. The availability of these dierent worldviews and individuals’ emotions em-
powers sentiment analysis. However, sentiment analysis becomes even more challenging due to a
scarcity of standardized labeled data in the Bangla NLP domain. The majority of the existing Bangla
research has relied on models of deep learning that signicantly focus on context-independent word
embeddings, such as Word2Vec, GloVe, and fastText, in which each word has a xed representation
irrespective of its context. Meanwhile, context-based pre-trained language models such as BERT have
recently revolutionized the state of natural language processing. In this work, we utilized BERT’s
transfer learning ability to a deep integrated model CNN-BiLSTM for enhanced performance of
decision-making in sentiment analysis. In addition, we also introduced the ability of transfer learning
to classical machine learning algorithms for the performance comparison of CNN-BiLSTM. Addi-
tionally, we explore various word embedding techniques, such as Word2Vec, GloVe, and fastText,
and compare their performance to the BERT transfer learning strategy. As a result, we have shown
a state-of-the-art binary classication performance for Bangla sentiment analysis that signicantly
outperforms all embedding and algorithms.
Keywords:
sentiment analysis; Bangla-BERT; transfer learning; transformer; word embedding; Bangla
NLP
1. Introduction
Sentiment classication is the process of examining a piece of text to forecast how an
individual’s aitude toward an occurrence or perspective will be oriented. The sentiment is
usually analyzed based on text polarity. Typically, a sentiment classier categorizes positive,
negative, or neutral [
1
]. Sentiment extraction is the backbone of sentiment categorization,
and considerable study has been conducted. The next crucial step is sentiment mining,
which has increased tremendously in recent years in line with the growth of textual data
worldwide. People now share their ideas electronically on various topics, including online
product reviews, book or lm studies, and political commentary. As a result, evaluating
diverse viewpoints becomes essential for interpreting people’s intentions. In general,
Sensors 2022, 22, 4157. https://doi.org/10.3390/s22114157 https://www.mdpi.com/journal/sensors
