A Systematic Literature Review of Topic Modeling Techniques in User Reviews
Downloads
Background: The escalating volume of user review data is necessitating automated methods for extracting valuable insights. Topic modeling was a vital method for understanding key discussions and user opinions. However, there was no comprehensive analysis of the scientific work specifically on topic modeling applied to user review datasets, including its main applications and a comparative analysis of the strengths and limitations of identified methods. This study addressed the gap by characterizing the scientific discussion, identifying potential directions, and exploring currently underutilized application areas within the context of user review analysis.
Objective: This study aimed to recognize the implementation trend of topic modeling in various areas and to comprehend the methodology that could be applied to the user review dataset.
Methods: A systematic literature review (SLR) was adopted by implementing Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines within six-year spans, narrowing 1746 to 28 selected primary studies.
Results: The underlying insight was that user reviews had been critical as the primary data for topic modeling in analyzing various applications. Digital banking and transportation applications were the sectors that received the greatest attention. In this context, Latent Dirichlet Allocation (LDA) was the most extensively used method, with a focus on overcoming its limitations by incorporating additional strategies into LDA-based models.
Conclusion: The bibliometric analysis and mapping study practically contributed as a reference when assessing the dominant topic in similar app categories and topic modeling algorithms. Furthermore, this study comprehensively analyzed various topic modeling algorithms, presenting both the strengths and weaknesses of informed selection in relevant applications. Considering the keywords cluster analysis, service quality could be adopted based on the output of the topic modeling.
Keywords: Topic modeling, User review, Systematic literature review, Bibliometric analysis
A. Abdelrazek, Y. Eid, E. Gawish, W. Medhat, and A. Hassan, “Topic modeling algorithms and applications: A survey,” Inf. Syst., vol. 112, p. 102131, 2023, doi: 10.1016/j.is.2022.102131.
A. M. Grisales, S. Robledo, and M. Zuluaga, “Topic Modeling: Perspectives From a Literature Review,” IEEE Access, vol. 11, no. November 2022, pp. 4066–4078, 2023, doi: 10.1109/ACCESS.2022.3232939.
D. Li, K. Wu, and V. L. C. Lei, “Applying Topic Modeling to Literary Analysis : A Review,” Digit. Stud. Lang. Lit., vol. 1, pp. 113–141, 2024, doi: 10.1515/dsll-2024-0010.
B. Ozyurt and M. A. Akcayol, “A new topic modeling based approach for aspect extraction in aspect based sentiment analysis: SS-LDA,” Expert Syst. Appl., vol. 168, p. 114231, 2021, doi: 10.1016/j.eswa.2020.114231.
L. George and P. Sumathy, “An integrated clustering and BERT framework for improved topic modeling,” Int. J. Inf. Technol., vol. 15, no. 4, pp. 2187–2195, 2023, doi: 10.1007/s41870-023-01268-w.
N. Genc-Nayebi and A. Abran, “A systematic literature review: Opinion mining studies from mobile app store user reviews,” J. Syst. Softw., vol. 125, pp. 207–219, Mar. 2017, doi: 10.1016/J.JSS.2016.11.027.
N. Jha and A. Mahmoud, “Mining non-functional requirements from App store reviews,” Empir. Softw. Eng., vol. 24, no. 6, pp. 3659–3695, 2019, doi: 10.1007/s10664-019-09716-7.
J. Dąbrowski, E. Letier, A. Perini, and A. Susi, “Mining and searching app reviews for requirements engineering: Evaluation and replication studies,” Inf. Syst., vol. 114, p. 102181, 2023, doi: 10.1016/j.is.2023.102181.
Y. Chen, Z. Peng, S. H. Kim, and C. W. Choi, “What We Can Do and Cannot Do with Topic Modeling: A Systematic Review,” Commun. Methods Meas., vol. 17, no. 2, pp. 111–130, 2023, doi: 10.1080/19312458.2023.2167965.
O. Grljević, M. Marić, and R. Božić, “Exploring Mobile Application User Experience Through Topic Modeling,” Sustain., vol. 17, no. 3, 2025, doi: 10.3390/su17031109.
S. Mankad, S. Hu, and A. Gopal, “Single stage prediction with embedded topic modeling of online reviews for mobile app management,” Ann. Appl. Stat., vol. 12, no. 4, pp. 2279–2311, 2018, doi: 10.1214/18-AOAS1152.
R. Wulandari and A. N. Hidayanto, “Measuring contact tracing service quality using sentiment analysis: a case study of PeduliLindungi Indonesia,” Qual. Quant., 2023, doi: 10.1007/s11135-023-01695-8.
L. Çallı, “Exploring mobile banking adoption and service quality features through user-generated content: the application of a topic modeling approach to Google Play Store reviews,” Int. J. Bank Mark., vol. 41, no. 2, pp. 428–454, 2023, doi: 10.1108/IJBM-08-2022-0351.
Y. Chen, H. Zhang, R. Liu, Z. Ye, and J. Lin, “Experimental explorations on short text topic mining between LDA and NMF based Schemes,” Knowledge-Based Syst., vol. 163, pp. 1–13, Jan. 2019, doi: 10.1016/J.KNOSYS.2018.08.011.
W. Chen, F. Rabhi, W. Liao, and I. Al-Qudah, “Leveraging State-of-the-Art Topic Modeling for News Impact Analysis on Financial Markets: A Comparative Study,” Electron., vol. 12, no. 12, p. 2605, 2023, doi: 10.3390/electronics12122605.
G. Papadia, M. Pacella, M. Perrone, and V. Giliberti, “A Comparison of Different Topic Modeling Methods through a Real Case Study of Italian Customer Care,” Algorithms, vol. 16, no. 2, pp. 1–19, 2023, doi: 10.3390/a16020094.
J. Dąbrowski, E. Letier, A. Perini, and A. Susi, “Analysing app reviews for software engineering: a systematic literature review,” Empir. Softw. Eng., vol. 27, no. 2, 2022, doi: 10.1007/s10664-021-10065-7.
J. J. C. Aman and J. Smith-Colin, “Application of crowdsourced data to infer user satisfaction with Mobility as a Service (MaaS),” Transp. Res. Interdiscip. Perspect., vol. 15, p. 100672, 2022, doi: https://doi.org/10.1016/j.trip.2022.100672.
O. Haggag, J. Grundy, M. Abdelrazek, and S. Haggag, “A large scale analysis of mHealth app user reviews,” Empir. Softw. Eng., vol. 27, no. 7, p. 196, 2022, doi: 10.1007/s10664-022-10222-6.
C. C. Silva, M. Galster, and F. Gilson, “Topic modeling in software engineering research,” Empir. Softw. Eng., vol. 26, no. 6, p. 120, 2021, doi: 10.1007/s10664-021-10026-0.
C. D. P. Laureate, W. Buntine, and H. Linger, “A systematic review of the use of topic models for short text social media analysis,” Artif. Intell. Rev., vol. 56, no. 12, pp. 14223–14255, 2023, doi: 10.1007/s10462-023-10471-x.
W. S. Wibowo and S. Yazid, “Unveiling Roadblocks and Mapping Solutions for Blockchain Adoption by Governments: A Systematic Literature Review,” Interdiscip. J. Information, Knowledge, Manag., vol. 18, no. June, pp. 547–581, 2023, doi: 10.28945/5186.
B. Kitchenham and P. Brereton, “A systematic review of systematic review process research in software engineering,” Inf. Softw. Technol., vol. 55, no. 12, pp. 2049–2075, 2013, doi: 10.1016/j.infsof.2013.07.010.
S. Castillo and P. Grbovic, “The APISSER Methodology for Systematic Literature Reviews in Engineering,” IEEE Access, vol. 10, pp. 23700–23707, 2022, doi: 10.1109/ACCESS.2022.3148206.
M. A. W. P. Rahmadhan, D. I. Sensuse, R. R. Suryono, and Kautsarina, “Trends and Applications of Gamification in E-Commerce : A Systematic Literature Review,” J. Inf. Syst. Eng. Bus. Intell., vol. 9, no. 1, pp. 28–37, 2023, doi: 10.20473/jisebi.9.1.28-37.
N. Z. Maharani, S. S. Kurniawan, D. I. Sensuse, I. Eitiveni, E. H. Purwaningsih, and D. S. Hidayat, “Motivations and Potential Solutions in Developing a Knowledge Management System for Organization at Higher Education : A Systematic Literature Review,” J. Inf. Syst. Eng. Bus. Intell., vol. 10, no. 2, pp. 270–289, 2024.
H. Ahmad, M. Ahtazaz, A. Adnan, and N. Mian, “Trends in publishing blockchain surveys : a bibliometric perspective,” Int. J. Inf. Secur., vol. 22, no. 2, pp. 511–523, 2023, doi: 10.1007/s10207-022-00653-z.
Y. M. Guo et al., “A bibliometric analysis and visualization of blockchain,” Futur. Gener. Comput. Syst., vol. 116, pp. 316–332, 2021, doi: 10.1016/j.future.2020.10.023.
I. Budi and R. R. Suryono, “Application of named entity recognition method for Indonesian datasets : a review,” Bull. Electr. Eng. Informatics, vol. 12, no. 2, pp. 969–978, 2023, doi: 10.11591/eei.v12i2.4529.
L. C. Cheng and L. R. Sharmayne, “Analysing Digital Banking Reviews Using Text Mining,” in Proceedings of the 12th IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 2021, pp. 914–918. doi: 10.1109/ASONAM49781.2020.9381429.
A. P. Darko, D. Liang, Z. Xu, K. Agbodah, and S. Obiora, “A novel multi-attribute decision-making for ranking mobile payment services using online consumer reviews,” Expert Syst. Appl., vol. 213, no. PC, p. 119262, 2023, doi: 10.1016/j.eswa.2022.119262.
A. Hussain, A. Hannan, and M. Shafiq, “Exploring mobile banking service quality dimensions in Pakistan: a text mining approach,” Int. J. Bank Mark., vol. 41, no. 3, pp. 601–618, Jan. 2023, doi: 10.1108/IJBM-08-2022-0379.
G. Shams, M. A. Rehman, S. Samad, and E. L. Oikarinen, “Exploring customer’s mobile banking experiences and expectations among generations X, Y and Z,” J. Financ. Serv. Mark., vol. 25, no. 1–2, pp. 1–13, 2020, doi: 10.1057/s41264-020-00071-z.
M. Tushev, F. Ebrahimi, and A. Mahmoud, “Domain-Specific Analysis of Mobile App Reviews Using Keyword-Assisted Topic Models,” in Proceedings of the 44th International Conference on Software Engineering, 2022, pp. 762–773. doi: 10.1145/3510003.3510201.
B. A. Çallı and Ç. Ediz, “Top concerns of user experiences in Metaverse games: A text-mining based approach,” Entertain. Comput., vol. 46, no. February, 2023, doi: 10.1016/j.entcom.2023.100576.
A. Kumar, S. Chakraborty, and P. K. Bala, “Text mining approach to explore determinants of grocery mobile app satisfaction using online customer reviews,” J. Retail. Consum. Serv., vol. 73, p. 103363, 2023, doi: https://doi.org/10.1016/j.jretconser.2023.103363.
C. I. Ossai and N. Wickramasinghe, “Automatic user sentiments extraction from diabetes mobile apps – An evaluation of reviews with machine learning,” Informatics Heal. Soc. Care, vol. 48, no. 3, pp. 211–230, Jul. 2023, doi: 10.1080/17538157.2022.2097083.
U. Yaqub, T. Saleem, and S. Zaman, “Analysis of COVID-19 Gov PK app user reviews to determine online privacy concerns of Pakistani citizens,” Glob. Knowledge, Mem. Commun., Nov. 2022, doi: 10.1108/GKMC-10-2022-0230.
M. Eiband, S. T. Völkel, D. Buschek, S. Cook, and H. Hussmann, “A Method and Analysis to Elicit User-Reported Problems in Intelligent Everyday Applications,” ACM Trans. Interact. Intell. Syst., vol. 10, no. 4, Nov. 2020, doi: 10.1145/3370927.
F. Viegas et al., “CluWords: Exploiting Semantic Word Clustering Representation for Enhanced Topic Modeling,” in Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, 2019, pp. 753–761. doi: 10.1145/3289600.3291032.
B. Chembakottu, H. Li, and F. Khomh, “A large-scale exploratory study of android sports apps in the google play store,” Inf. Softw. Technol., vol. 164, p. 107321, 2023, doi: 10.1016/j.infsof.2023.107321.
Y. Zhang, X. Shi, Z. Abdul-Hamid, D. Li, X. Zhang, and Z. Shen, “Factors influencing crowdsourcing riders’ satisfaction based on online comments on real-time logistics platform,” Transp. Lett., vol. 15, no. 5, pp. 363–374, May 2023, doi: 10.1080/19427867.2022.2052643.
V. H. Pranatawijaya, N. N. K. Sari, R. A. Rahman, E. Christian, and S. Geges, “Unveiling User Sentiment: Aspect-Based Analysis and Topic Modeling of Ride-Hailing and Google Play App Reviews,” J. Inf. Syst. Eng. Bus. Intell., vol. 10, no. 3, pp. 328–339, 2024, doi: 10.20473/jisebi.10.3.328-339.
R. A. Masrury, Fannisa, and A. Alamsyah, “Analyzing Tourism Mobile Applications Perceived Quality using Sentiment Analysis and Topic Modeling,” in 2019 7th International Conference on Information and Communication Technology (ICoICT), 2019, pp. 1–6. doi: 10.1109/ICoICT.2019.8835255.
C. Tao, H. Guo, and Z. Huang, “Identifying security issues for mobile applications based on user review summarization,” Inf. Softw. Technol., vol. 122, p. 106290, 2020, doi: https://doi.org/10.1016/j.infsof.2020.106290.
F. Viegas et al., “Semantically-Enhanced Topic Modeling,” in Proceedings of the 27th ACM International Conference on Information and Knowledge Management, 2018, pp. 893–902. doi: 10.1145/3269206.3271797.
T. Yang, C. Gao, J. Zang, D. Lo, and M. Lyu, “TOUR: Dynamic Topic and Sentiment Analysis of User Reviews for Assisting App Release,” in Companion Proceedings of the Web Conference 2021, 2021, pp. 708–712. doi: 10.1145/3442442.3458612.
A. P. Darko, D. Liang, Z. Xu, K. Agbodah, and S. Obiora, “A novel multi-attribute decision-making for ranking mobile payment services using online consumer reviews,” Expert Syst. Appl., vol. 213, p. 119262, 2023, doi: https://doi.org/10.1016/j.eswa.2022.119262.
X. Shi and J. Li, “Key factors for instant logistics platforms to attract and retain couriers: An analysis based on online comments,” Res. Transp. Bus. Manag., vol. 50, p. 101031, Oct. 2023, doi: 10.1016/j.rtbm.2023.101031.
J. J. C. Aman, J. Smith-Colin, and W. Zhang, “Listen to E-scooter riders: Mining rider satisfaction factors from app store reviews,” Transp. Res. Part D Transp. Environ., vol. 95, p. 102856, 2021, doi: 10.1016/j.trd.2021.102856.
S. F. Verkijika and B. N. Neneh, “Standing up for or against: A text-mining study on the recommendation of mobile payment apps,” J. Retail. Consum. Serv., vol. 63, p. 102743, 2021, doi: https://doi.org/10.1016/j.jretconser.2021.102743.
D. R. K. Raja and S. Pushpa, “Diversifying personalized mobile multimedia application recommendations through the Latent Dirichlet Allocation and clustering optimization,” Multimed. Tools Appl., vol. 78, no. 17, pp. 24047–24066, 2019, doi: 10.1007/s11042-019-7190-7.
M. A. Hadi and F. H. Fard, “ReviewViz: Assisting Developers Perform Empirical Study on Energy Consumption Related Reviews for Mobile Applications,” in 2020 IEEE/ACM 7th International Conference on Mobile Software Engineering and Systems (MOBILESoft), 2020, pp. 27–30. doi: 10.1145/3387905.3388605.
C. Gao et al., “Emerging App Issue Identification via Online Joint Sentiment-Topic Tracing,” IEEE Trans. Softw. Eng., vol. 48, no. 8, pp. 3025–3043, 2022, doi: 10.1109/TSE.2021.3076179.
M. Hatamian, J. Serna, and K. Rannenberg, “Revealing the unrevealed: Mining smartphone users privacy perception on app markets,” Comput. Secur., vol. 83, pp. 332–353, 2019, doi: https://doi.org/10.1016/j.cose.2019.02.010.
D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent Dirichlet Allocation: Extracting Topics from Software Engineering Data,” Art Sci. Anal. Softw. Data, vol. 3, pp. 139–159, 2015, doi: 10.1016/B978-0-12-411519-4.00006-9.
C. Gao, J. Zeng, M. R. Lyu, and I. King, “Online App Review Analysis for Identifying Emerging Issues,” Proc. - Int. Conf. Softw. Eng., vol. 2018-Janua, pp. 48–58, 2018, doi: 10.1145/3180155.3180218.
M. A. Hadi and F. H. Fard, “AOBTM: Adaptive Online Biterm Topic Modeling for Version Sensitive Short-texts Analysis,” Proc. - 2020 IEEE Int. Conf. Softw. Maint. Evol. ICSME 2020, pp. 593–604, 2020, doi: 10.1109/ICSME46990.2020.00062.
X. Cheng, X. Yan, Y. Lan, and J. Guo, “BTM: Topic modeling over short texts,” IEEE Trans. Knowl. Data Eng., vol. 26, no. 12, pp. 2928–2941, 2014, doi: 10.1109/TKDE.2014.2313872.
H. Jelodar et al., “Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey,” Multimed. Tools Appl., vol. 78, no. 11, pp. 15169–15211, 2019, doi: 10.1007/s11042-018-6894-4.
A. Gupta and H. Fatima, “Topic Modeling in Healthcare: A Survey Study,” NeuroQuantology, vol. 20, no. 11, pp. 6214–6221, 2022, doi: 10.14704/NQ.2022.20.11.NQ66619.
T. N. Saifana and N. Wilantika, “User Perception towards Telemedicine Before and During COVID-19,” in 2022 International Conference on Advanced Computer Science and Information Systems (ICACSIS), 2022, pp. 123–130. doi: 10.1109/ICACSIS56558.2022.9923475
Copyright (c) 2025 The Authors. Published by Universitas Airlangga.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
All accepted papers will be published under a Creative Commons Attribution 4.0 International (CC BY 4.0) License. Authors retain copyright and grant the journal right of first publication. CC-BY Licenced means lets others to Share (copy and redistribute the material in any medium or format) and Adapt (remix, transform, and build upon the material for any purpose, even commercially).
