Unveiling User Sentiment: Aspect-Based Analysis and Topic Modeling of Ride-Hailing and Google Play App Reviews
Downloads
Background: Mobile app usage is increasing in the digital age, with Ride-Hailing app becoming the primary example of this trend. To obtain valuable understanding of how people perceive and interact with mobile app, user reviews on platforms such as Google Play are usually analyzed. This analysis can assist developers to identify areas for improvement in both Ride-hailing and Google Play App. A promising method that can be used to analyze user perception in this instance is Aspect-Based Sentiment Analysis (ABSA).
Objective: This research aimed to apply ABSA to user reviews using Bidirectional Encoder Representations from Transformers (BERT) models. In this context, aspect identification and topic modeling were performed by using Latent Dirichlet Allocation (LDA). The model extracted topics from the reviews and used Generative Artificial Intelligence (GenAI) to define the aspects of the topics to further enhance the analysis. For consistency and accuracy, the method included sentiment annotation by a human annotator.
Methods: A total of two datasets were used in this research, with the first collected by scraping user reviews of Ride-Hailing App while the second was obtained from Kaggle, and to identify relevant topics, modeling was performed using LDA. These topics were then categorized into aspects using GenAI, covering areas, such as customer experience, service, payment, app features, task management, and event management. Subsequently, sentiment labeling was conducted using human annotators to provide a reliable baseline. BERT model was then used to classify sentiment with aspect hints, and the evaluation included calculations of accuracy, precision, recall, and F1-score.
Results: The results showed that BERT model achieved the highest accuracy of 97% in sentiment analysis across all datasets.
Conclusion: This research provided valuable understanding of user experience and established a strong ABSA framework for analyzing user reviews using LDA, Aspect Annotation, GenAI, and BERT sentiment models. Future research could expand this method to other app categories and incorporate real-time ABSA for continuous monitoring and dynamic feedback.
Keywords: User Reviews, Aspect-Based Sentiment Analysis (ABSA), Sentiment Analysis, Topic Modeling, Generative Artificial Intelligence (GenAI)
R. Bose, R. K. Dey, S. Roy, and D. Sarddar, “Sentiment analysis on online product reviews,” in Information and Communication Technology for Sustainable Development, 2020, pp. 559–569. doi: 10.1007/978-981-13-7166-0_56.
A. Hermawan, I. Jowensen, J. Junaedi, and Edy, “Implementasi text-mining untuk analisis sentimen pada twitter dengan algoritma Support Vector Machine,” Jurnal Sains dan Teknologi, vol. 12, no. 1, pp. 129–137, Apr. 2023, doi: 10.23887/jstundiksha.v12i1.52358.
P. H. Prastyo, A. S. Sumi, A. W. Dian, and A. E. Permanasari, “Tweets responding to the Indonesian government’s handling of Covid-19: sentiment analysis using SVM with normalized poly kernel,” Journal of Information Systems Engineering and Business Intelligence, vol. 6, no. 2, p. 112, Oct. 2020, doi: 10.20473/jisebi.6.2.112-122.
C. A. Bahri and L. H. Suadaa, “Aspect-based sentiment analysis in Bromo Tengger Semeru National Park Indonesia based on google maps user reviews,” Indonesian Journal of Computing and Cybernetics Systems, vol. 17, no. 1, p. 79, Feb. 2023, doi: 10.22146/ijccs.77354.
S. A. P. Perdana, T. B. Aji, and R. Ferdiana, “Aspect category classification with machine learning approach using Indonesian language dataset,” Jurnal Nasional Teknik Elektro dan Teknologi Informasi, vol. 10, no. 3, pp. 229-235, 2021, doi: 10.22146/jnteti.v10i3.1819.
I. A. Kandhro, F. Ali, M. Uddin, A. Kehar, and S. Manickam, “Exploring aspect-based sentiment analysis: An in-depth review of current methods and prospects for advancement,” Knowl. Inf. Syst., vol. 66, no. 7, pp. 3639–3669, Jul. 2024, doi: 10.1007/s10115-024-02104-8.
R. Bharathi, R. Bhavani, and R. Priya, “Leveraging deep learning models for automated aspect based sentiment analysis and classification,” SSRG International Journal of Electrical and Electronics Engineering, vol. 10, no. 5, pp. 120–130, May 2023, doi: 10.14445/23488379/IJEEE-V10I5P111.
S. Cahyaningtyas, D. Hatta Fudholi, and A. Fathan Hidayatullah, “Deep learning for aspect-based sentiment analysis on Indonesian hotels reviews,” Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, Aug. 2021, doi: 10.22219/kinetik.v6i3.1300.
Salsabila, S. M. P. Tyas, Y. Romadhona, and D. Purwitasari, “Aspect-based sentiment and correlation-based emotion detection on tweets for understanding public opinion of Covid-19.,” Journal of Information Systems Engineering and Business Intelligence, vol. 9, no. 1, pp. 84–94, 2023, doi: 10.20473/jisebi.9.1.84-94.
W. A. Nawang Sari and H. Dwi Purnomo, “Topic modeling using the latent dirichlet allocation method on Wikipedia pandemic Covid-19 data in Indonesia,” Jurnal Teknik Informatika, vol. 3, no. 5, pp. 1223–1230, Oct. 2022, doi: 10.20884/1.jutif.2022.3.5.321.
F. Rashif, G. Ihza Perwira Nirvana, M. Alif Noor, and N. Aini Rakhmawati, “Implementasi LDA untuk pengelompokan topik cuitan akun bot Twitter bertagar #Covid-19,” Cogito Smart Journal, vol. 7, no. 1, 2021.
V. S. Anoop and S. Asharaf, “Aspect-oriented sentiment analysis: A topic modeling-powered approach,” Journal of Intelligent Systems, vol. 29, no. 1, pp. 1166–1178, Jan. 2020, doi: 10.1515/jisys-2018-0299.
R. Gozalo-Brizuela and E. C. Garrido-Merchán, “A survey of generative AI applications,” Jun. 2023, [Online]. Available: http://arxiv.org/abs/2306.02781
P. Denny et al., “Computing education in the era of Generative AI,” Jun. 2023, [Online]. Available: http://arxiv.org/abs/2306.02608
Gemini Team et al., “Gemini: A family of highly capable multimodal models,” Dec. 2023, [Online]. Available: http://arxiv.org/abs/2312.11805
F. Rodríguez-Torres, J. F. Martínez-Trinidad, and J. A. Carrasco-Ochoa, “An oversampling method for class imbalance problems on large datasets,” Applied Sciences, vol. 12, no. 7, p. 3424, Apr. 2022, doi: 10.3390/app12073424.
P. Gnip, L. Vokorokos, and P. Drotár, “Selective oversampling approach for strongly imbalanced data,” PeerJ Comput Sci, vol. 7, pp. 1–22, 2021, doi: 10.7717/PEERJ-CS.604.
J. Z. Maitama, N. Idris, A. Abdi, and A. T. Bimba, “Aspect extraction in sentiment analysis based on emotional affect using supervised approach,” in 2021 4th International Conference on Artificial Intelligence and Big Data, ICAIBD 2021, Institute of Electrical and Electronics Engineers Inc., May 2021, pp. 372–376. doi: 10.1109/ICAIBD51990.2021.9458996.
O. J. Prasad, S. Nandi, V. Dogra, and D. S. Diwakar, “A systematic review of NLP methods for sentiment classification of online news articles,” in 2023 14th International Conference on Computing Communication and Networking Technologies, ICCCNT 2023, Institute of Electrical and Electronics Engineers Inc., 2023. doi: 10.1109/ICCCNT56998.2023.10308056.
T. Sai Aparna, K. Simran, B. Premjith, and K. P. Soman, “Aspect-based sentiment analysis in Hindi: Comparison of machine/deep learning algorithms,” 2021, pp. 81–91. doi: 10.1007/978-981-33-4305-4_7.
M. P. Geetha and D. Karthika Renuka, “Improving the performance of aspect based sentiment analysis using fine-tuned BERT Base Uncased model,” International Journal of Intelligent Networks, vol. 2, pp. 64–69, Jan. 2021, doi: 10.1016/j.ijin.2021.06.005.
T. H. Khan and S. Ridhorkar, “Improving the efficiency of aspect-based sentiment analysis using ensemble deep learning,” Int J Intell Syst Appl Eng, vol. 12, no. 10s, pp. 526–538, Jan. 2024.
Q. Liu, Z. Gao, B. Liu, and Y. Zhang, “Automated rule selection for opinion target extraction,” Knowledge-Based Systems, vol. 104, pp. 74–88, Jul. 2016, doi: 10.1016/j.knosys.2016.04.010.
W. Zhang, X. Li, Y. Deng, L. Bing, and W. Lam, “A survey on aspect-based sentiment analysis: Tasks, methods, and challenges,” Mar. 2022, [Online]. Available: http://arxiv.org/abs/2203.01054
A. A. Bakar, LK. Soon, and HN. Goh, “An exploratory study on Latent-Dirichlet Allocation Models for aspect identification on short sentences,” in Computational Science and Tecnology: 4th ICCST 2017, 2018, pp. 314–323. doi: 10.1007/978-981-10-8276-4_30.
M. P. Abraham, K. R. Udaya Kumar Reddy, and S. Abraham, “Aspect based sentiment analysis system using machine learning techniques,” in 2023 IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics (DISCOVER), IEEE, Oct. 2023, pp. 25–30. doi: 10.1109/DISCOVER58830.2023.10316716.
J. W. Iskandar and Y. Nataliani, “Perbandingan Naïve Bayes, SVM, dan k-NN untuk analisis sentimen gadget berbasis aspek,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 5, no. 6, pp. 1120–1126, Dec. 2021, doi: 10.29207/resti.v5i6.3588.
D. Deviacita et al., “Implementasi web scraping untuk pengambilan data pada situs marketplace,” Jurnal Sistem dan Teknologi Informasi (JustIN), vol. 7, no. 4, 2019, doi: 10.26418/justin.v7i4.30930.
A. Ariyo Munandar and C. Edi Widodo, “Sentimen analisis aplikasi belajar online menggunakan klasifikasi SVM,” Journal of Information Technology and Computer Science (JOINTECS), vol. 7, no. 2, pp. 77–84, 2023, doi: 10.31328/jointecs.v8i2.4747.
S. Patel, “Google Play apps reviews,” Kaggle, 2023, Accessed: Feb. 20, 2024. [Online]. Available: https://www.kaggle.com/datasets/therealsampat/google-play-apps-reviews
L. Hermawan and M. B. Ismiati, “Pembelajaran text preprocessing berbasis simulator untuk mata kuliah information retrieval,” Jurnal Transformatika, vol. 17, no. 2, pp. 188–199, 2020, doi: 10.26623/transformatika.v17i2.1705.
G. Rau and Y. S. Shih, “Evaluation of Cohen’s kappa and other measures of inter-rater agreement for genre analysis and other nominal data,” J Engl Acad Purp, vol. 53, p. 101026, Sep. 2021, doi: 10.1016/j.jeap.2021.101026.
Z. Yang and M. Zhou, “Kappa statistic for clustered matched-pair data,” Stat Med, vol. 33, no. 15, pp. 2612–2633, Jul. 2014, doi: 10.1002/sim.6113.
P. Czodrowski, “Count on kappa,” J Comput Aided Mol Des, vol. 28, no. 11, pp. 1049–1055, 2014, doi: 10.1007/s10822-014-9759-6.
S. Y. Ng, K. M. Lim, C. P. Lee, and J. Y. Lim, “Sentiment analysis using DistilBERT,” in 2023 IEEE 11th Conference on Systems, Process & Control (ICSPC), IEEE, Dec. 2023, pp. 84–89. doi: 10.1109/ICSPC59664.2023.10420272.
Y. Yang, C. Miller, P. Jiang, and A. Moghtaderi, “A case study of multi-class classification with diversified precision recall requirements for query disambiguation,” in SIGIR 2020 - Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Association for Computing Machinery, Inc, Jul. 2020, pp. 1633–1636. doi: 10.1145/3397271.3401315.
Copyright (c) 2024 The Authors. Published by Universitas Airlangga.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
All accepted papers will be published under a Creative Commons Attribution 4.0 International (CC BY 4.0) License. Authors retain copyright and grant the journal right of first publication. CC-BY Licenced means lets others to Share (copy and redistribute the material in any medium or format) and Adapt (remix, transform, and build upon the material for any purpose, even commercially).
