Domain-Specific Fine-Tuning of IndoBERT for Aspect-Based Sentiment Analysis in Indonesian Travel User-Generated Content
Downloads
Background: Aspect-based sentiment analysis (ABSA) is essential in extracting meaningful insights from user-generated content (UGC) in various domains. In tourism, UGC such as Google Reviews offers essential feedback, but the challenges associated with processing in Indonesian language, including the unique linguistic characteristics, pose difficulties for automatic sentiment, and aspect detection. Recent advancements in transformer-based models, such as BERT, have shown great potential in addressing these challenges by providing context-aware embeddings.
Objective: This research aimed to fine-tune IndoBERT, a pre-trained Indonesian language model, to perform information extraction and key aspect detection from tourism-related UGC. The objective was to identify critical aspects of tourism reviews and classify their sentiments.
Methods: A dataset of 20,000 Google Reviews, focusing on 20 tourism destinations in DI Yogyakarta and Jawa Tengah, was collected and preprocessed. Multiple fine-tuning experiments were conducted, using a layer-freezing method by adjusting only the top layers of IndoBERT, while freezing others to determine the optimal configuration. The model's performance was evaluated based on validation loss, precision, recall, and F1-score in aspect detection and overall sentiment classification accuracy.
Results: The best-performing configuration involved freezing the last six layers and fine-tuning the top six layers of IndoBERT, yielding a validation loss of 0.324. The model achieved precision scores between 0.85 and 0.89 in aspect detection and an overall sentiment classification accuracy of 0.84. Error analysis revealed challenges in distinguishing neutral and negative sentiments and in handling reviews with multiple aspects or mixed sentiments.
Conclusion: The fine-tuned IndoBERT model effectively extracted key tourism aspects and classified sentiments from Indonesian UGC. While the model performed well in detecting strong sentiments, improvements are needed to handle neutral and mixed sentiments better. Future work will explore sentiment intensity analysis and aspect segmentation methods to enhance the model's performance.
Keywords: Aspect-Based Sentiment Analysis, Fine-tuning, IndoBERT, Sentiment Classification, Tourism Reviews, User-Generated Content
A. Fronzetti Colladon, B. Guardabascio, and R. Innarella, “Using social network and semantic analysis to analyze online travel forums and forecast tourism demand,” Decis Support Syst, vol. 123, p. 113075, Aug. 2019, doi: 10.1016/j.dss.2019.113075.
S. Barandoni, F. Chiarello, L. Cascone, E. Marrale, and S. Puccio, “Automating Customer Needs Analysis: A Comparative Study of Large Language Models in the Travel Industry,” Apr. 2024.
N. Raghunathan and K. Saravanakumar, “Challenges and Issues in Sentiment Analysis: A Comprehensive Survey,” IEEE Access, vol. 11, pp. 69626–69642, 2023, doi: 10.1109/ACCESS.2023.3293041.
W. Zhang, X. Li, Y. Deng, L. Bing, and W. Lam, “A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges,” IEEE Trans Knowl Data Eng, vol. 35, no. 11, pp. 11019–11038, Nov. 2023, doi: 10.1109/TKDE.2022.3230975.
A. Jain, A. Bansal, and S. Tomar, “Aspect-Based Sentiment Analysis of Online Reviews for Business Intelligence,” International Journal of Information Technologies and Systems Approach, vol. 15, no. 3, pp. 1–21, Aug. 2022, doi: 10.4018/IJITSA.307029.
N. Raghunathan and K. Saravanakumar, “Challenges and Issues in Sentiment Analysis: A Comprehensive Survey,” IEEE Access, vol. 11, pp. 69626–69642, 2023, doi: 10.1109/ACCESS.2023.3293041.
W. Zhang, X. Li, Y. Deng, L. Bing, and W. Lam, “A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges,” IEEE Trans Knowl Data Eng, vol. 35, no. 11, pp. 11019–11038, Nov. 2023, doi: 10.1109/TKDE.2022.3230975.
A. Jain, A. Bansal, and S. Tomar, “Aspect-Based Sentiment Analysis of Online Reviews for Business Intelligence,” International Journal of Information Technologies and Systems Approach, vol. 15, no. 3, pp. 1–21, Aug. 2022, doi: 10.4018/IJITSA.307029.
Q. Jiang, L. Chen, R. Xu, X. Ao, and M. Yang, “A Challenge Dataset and Effective Models for Aspect-Based Sentiment Analysis,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Stroudsburg, PA, USA: Association for Computational Linguistics, 2019, pp. 6279–6284. doi: 10.18653/v1/D19-1654.
A. Nazir, Y. Rao, L. Wu, and L. Sun, “Issues and Challenges of Aspect-based Sentiment Analysis: A Comprehensive Survey,” IEEE Trans Affect Comput, vol. 13, no. 2, pp. 845–863, Apr. 2022, doi: 10.1109/TAFFC.2020.2970399.
A. Vaswani et al., “Attention Is All You Need,” Jun. 2017, [Online]. Available: http://arxiv.org/abs/1706.03762
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language Models are Unsupervised Multitask Learners,” 2019.
J. Devlin, M.-W. Chang, K. Lee, K. T. Google, and A. I. Language, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Proceedings of the Conference, 2019, pp. 4171–4186. [Online]. Available: https://github.com/tensorflow/tensor2tensor
Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. Salakhutdinov, and Q. V Le, “XLNet: Generalized Autoregressive Pretraining for Language Understanding,” in 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), 2019. [Online]. Available: https://github.com/zihangdai/xlnet
B. Wilie et al., “IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding,” in Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, Stroudsburg, PA, USA: Association for Computational Linguistics, 2020, pp. 843–857. doi: 10.18653/v1/2020.aacl-main.85.
B. Wilie et al., “IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding,” in Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, Stroudsburg, PA, USA: Association for Computational Linguistics, 2020, pp. 843–857. doi: 10.18653/v1/2020.aacl-main.85.
B. Wilie et al., “IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding,” 2020. [Online]. Available: https://github.com/annisanurulazhar/absa-playground
M. Polignano, P. Basile, M. De Gemmis, G. Semeraro, and V. Basile, “ALBERTO: Italian BERT Language Understanding Model for NLP Challenging Tasks Based on Tweets,” in CEUR Workshop Proceedings, 2019. [Online]. Available: https://github.com/google-research/
R. Scheible, F. Thomczyk, P. Tippmann, V. Jaravine, and M. Boeker, “GottBERT: a pure German Language Model,” 2020. [Online]. Available: https://tblock.github.io/10kGNAD
C. Sun, X. Qiu, Y. Xu, and X. Huang, “How to Fine-Tune BERT for Text Classification?,” 2020. [Online]. Available: https://github.
Q. Thai Nguyen, T. Linh Nguyen, N. Hoang Luong, and Q. Hung Ngo, “Fine-Tuning BERT for Sentiment Analysis of Vietnamese Reviews,” in Proceedings - 2020 7th NAFOSTED Conference on Information and Computer Science, NICS 2020, 2020. [Online]. Available: https://github.com/google-research/bert
F. Koto, A. Rahimi, J. H. Lau, and T. Baldwin, “IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP,” Nov. 2020, [Online]. Available: http://arxiv.org/abs/2011.00677
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of the 2019 Conference of the North, Stroudsburg, PA, USA: Association for Computational Linguistics, 2019, pp. 4171–4186. doi: 10.18653/v1/N19-1423.
E. Yulianti, N. Pangestu, and M. A. Jiwanggi, “Enhanced TextRank using weighted word embedding for text summarization,” International Journal of Electrical and Computer Engineering (IJECE), vol. 13, no. 5, p. 5472, Oct. 2023, doi: 10.11591/ijece.v13i5.pp5472-5482.
N. K. Nissa and E. Yulianti, “Multi-label text classification of Indonesian customer reviews using bidirectional encoder representations from transformers language model,” International Journal of Electrical and Computer Engineering (IJECE), vol. 13, no. 5, p. 5641, Oct. 2023, doi: 10.11591/ijece.v13i5.pp5641-5652.
M. Polignano, V. Basile, P. Basile, M. de Gemmis, and G. Semeraro, “AlBERTo: Modeling Italian Social Media Language with BERT,” Italian Journal of Computational Linguistics, vol. 5, no. 2, pp. 11–31, Dec. 2019, doi: 10.4000/ijcol.472.
Q. T. Nguyen, T. L. Nguyen, N. H. Luong, and Q. H. Ngo, “Fine-Tuning BERT for Sentiment Analysis of Vietnamese Reviews,” in 2020 7th NAFOSTED Conference on Information and Computer Science (NICS), IEEE, Nov. 2020, pp. 302–307. doi: 10.1109/NICS51282.2020.9335899.
C. A. Bahri and L. H. Suadaa, “Aspect-Based Sentiment Analysis in Bromo Tengger Semeru National Park Indonesia Based on Google Maps User Reviews,” IJCCS (Indonesian Journal of Computing and Cybernetics Systems), vol. 17, no. 1, p. 79, Feb. 2023, doi: 10.22146/ijccs.77354.
F. Said and L. P. Manik, “Aspect-Based Sentiment Analysis on Indonesian Presidential Election Using Deep Learning,” Paradigma - Jurnal Komputer dan Informatika, vol. 24, no. 2, pp. 160–167, Sep. 2022, doi: 10.31294/paradigma.v24i2.1415.
A. Tiwari, K. Tewari, S. Dawar, A. Singh, and N. Rathee, “Comparative Analysis on Aspect-based Sentiment using BERT,” in 2023 7th International Conference on Computing Methodologies and Communication (ICCMC), IEEE, Feb. 2023, pp. 723–727. doi: 10.1109/ICCMC56507.2023.10084294.
C. Sun, X. Qiu, Y. Xu, and X. Huang, “How to Fine-Tune BERT for Text Classification?,” May 2019.
M. E. Peters et al., “Deep contextualized word representations,” Feb. 2018.
F. Said and L. P. Manik, “Aspect-Based Sentiment Analysis on Indonesian Presidential Election Using Deep Learning,” Paradigma - Jurnal Komputer dan Informatika, vol. 24, no. 2, pp. 160–167, Sep. 2022, doi: 10.31294/paradigma.v24i2.1415.
C. A. Bahri and L. H. Suadaa, “Aspect-Based Sentiment Analysis in Bromo Tengger Semeru National Park Indonesia Based on Google Maps User Reviews,” IJCCS (Indonesian Journal of Computing and Cybernetics Systems), vol. 17, no. 1, p. 79, Feb. 2023, doi: 10.22146/ijccs.77354.
C. Sun, X. Qiu, Y. Xu, and X. Huang, “How to Fine-Tune BERT for Text Classification?,” May 2019.
Y. Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” 2019. [Online]. Available: https://github.com/pytorch/fairseq
M. Mosbach, M. Andriushchenko, and D. Klakow, “On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines,” Jun. 2020.
N. Srivastava, G. Hinton, A. Krizhevsky, and R. Salakhutdinov, “Dropout: A Simple Way to Prevent Neural Networks from Overfitting,” 2014.
I. Loshchilov and F. Hutter, “Decoupled Weight Decay Regularization,” Nov. 2017.
A. N. Azhar and M. L. Khodra, “Fine-tuning Pretrained Multilingual BERT Model for Indonesian Aspect-based Sentiment Analysis,” in 2020 7th International Conference on Advance Informatics: Concepts, Theory and Applications (ICAICTA), IEEE, Sep. 2020, pp. 1–6. doi: 10.1109/ICAICTA49861.2020.9428882.
Copyright (c) 2025 The Authors. Published by Universitas Airlangga.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
All accepted papers will be published under a Creative Commons Attribution 4.0 International (CC BY 4.0) License. Authors retain copyright and grant the journal right of first publication. CC-BY Licenced means lets others to Share (copy and redistribute the material in any medium or format) and Adapt (remix, transform, and build upon the material for any purpose, even commercially).
