Optimizing Support Vector Machine Performance for Parkinson's Disease Diagnosis Using GridSearchCV and PCA-Based Feature Extraction
Downloads
Background: Parkinson's disease (PD) is a critical neurodegenerative disorder affecting the central nervous system and often causing impaired movement and cognitive function in patients. In addition, its diagnosis in the early stages requires a complex and time-consuming process because all existing tests such as electroencephalography or blood examinations lack effectiveness and accuracy. Several studies explored PD prediction using sound, with a specific focus on the development of classification models to enhance accuracy. The majority of these neglected crucial aspects including feature extraction and proper parameter tuning, leading to low accuracy.
Objective: This study aims to optimize performance of voice-based PD prediction through feature extraction, with the goal of reducing data dimensions and improving model computational efficiency. Additionally, appropriate parameters will be selected for enhancement of the ability of the model to identify both PD cases and healthy individuals.
Methods: The proposed new model applied an OpenML dataset comprising voice recordings from 31 individuals, namely 23 PD patients and 8 healthy participants. The experimental process included the initial use of the SVM algorithm, followed by implementing PCA for feature extraction to enhance machine learning accuracy. Subsequently, data balancing with SMOTE was conducted, and GridSearchCV was used to identify the best parameter combination based on the predicted model characteristics.
Result: Evaluation of the proposed model showed an impressive accuracy of 97.44%, sensitivity of 100%, and specificity of 85.71%. This excellent result was achieved with a limited dataset and a 10-fold cross-validation tuning, rendering the model sensitive to the training data.
Conclusion: This study successfully enhanced the prediction model accuracy through the SVM+PCA+GridSearchCV+CV method. However, future investigations should consider an appropriate number of folds for a small dataset, explore alternative cross-validation methods, and expand the dataset to enhance model generalizability.
Keywords: GridSearchCV, Parkinson Disaese, SVM, PCA, SMOTE, Voice/Speech
M. S. Alzubaidi et al., "The role of neural network for the detection of parkinson's disease: A scoping review,” Healthcare (Switzerland), vol. 9, no. 6, pp. 1–20, 2021, doi: 10.3390/healthcare9060740.
Z. Karapinar Senturk, "Early diagnosis of Parkinson's disease using machine learning algorithms,” Med Hypotheses, vol. 138, no. January, p. 109603, 2020, doi: 10.1016/j.mehy.2020.109603.
H. Gunduz, "Deep Learning-Based Parkinson's Disease Classification Using Vocal Feature Sets,” IEEE Access, vol. 7, pp. 115540–115551, 2019, doi: 10.1109/ACCESS.2019.2936564.
M. Alfonsetti, V. Castelli, and M. D'angelo, "Are We What We Eat? Impact of Diet on the Gut–Brain Axis in Parkinson's Disease,” Nutrients, vol. 14, no. 2, 2022, doi: 10.3390/nu14020380.
C. Quan, K. Ren, and Z. Luo, "A Deep Learning Based Method for Parkinson's Disease Detection Using Dynamic Features of Speech,” IEEE Access, vol. 9, pp. 10239–10252, 2021, doi: 10.1109/ACCESS.2021.3051432.
C. Kobylecki, "Update on the diagnosis and management of Parkinson's disease,” Clinical Medicine, Journal of the Royal College of Physicians of London, vol. 20, no. 4, pp. 393–398, 2020, doi: 10.7861/CLINMED.2020-0220.
N. Maserejian, L. Vinikoor-Imle, and A. Dilley, "Estimation of the 2020 Global Population of Parkinson's Disease (PD) [abstract],” Movement Disorders, vol. 35, p. 198, 2020.
R. Sheibani, E. Nikookar, and S. Alavi, "An ensemble method for diagnosis of Parkinson's disease based on voice measurements,” J Med Signals Sens, vol. 9, no. 4, pp. 221–226, 2019, doi: 10.4103/jmss.JMSS_57_18.
K. M. Alalayah, E. M. Senan, H. F. Atlam, I. A. Ahmed, and H. S. A. Shatnawi, "Automatic and Early Detection of Parkinson's Disease by Analyzing Acoustic Signals Using Classification Algorithms Based on Recursive Feature Elimination Method,” Diagnostics, vol. 13, no. 11, 2023, doi: 10.3390/diagnostics13111924.
A. M. Maitín, A. J. García-Tejedor, and J. P. R. Muñoz, "Machine learning approaches for detecting parkinson's disease from eeg analysis: A systematic review,” Applied Sciences (Switzerland), vol. 10, no. 23, pp. 1–21, 2020, doi: 10.3390/app10238662.
R. Mathur, V. Pathak, and D. Bandil, Emerging Trends in Expert Applications and Security, vol. 841. Springer Singapore, 2019. doi: 10.1007/978-981-13-2285-3.
J. Unjung and M. R. Ningsih, "Optimized Handwriting-based Parkinson's Disease Classification Using Ensemble Modeling and VGG19 Feature Extraction,” Scientific Journal of Informatics, vol. 10, no. 4, pp. 489–498, 2023.
D. Mata-Marín, J. A. Pineda-Pardo, J. A. Molina, L. Vela, F. Alonso-Frech, and I. Obeso, "Aberrant Salient and Corticolimbic Connectivity in Hypersexual Parkinson's Disease,” Brain Connectivity, vol. 11, no. 8, pp. 639–650, 2021, doi: 10.1089/brain.2020.0868.
C. Jatoth, E. Neelima, A. V. R. Mayuri, and S. R. Annaluri, "Effective monitoring and prediction of Parkinson disease in Smart Cities using intelligent health care system,” Microprocess Microsyst, vol. 92, no. June 2021, p. 104547, 2022, doi: 10.1016/j.micpro.2022.104547.
E. V. Altay and B. Alatas, "Association analysis of Parkinson disease with vocal change characteristics using multi-objective metaheuristic optimization,” Medical Hypotheses, vol. 141, no. March, p. 109722, 2020, doi: 10.1016/j.mehy.2020.109722.
P. Klumpp et al., "The phonetic footprint of Parkinson's disease,” Comput Speech Lang, vol. 72, no. July 2021, p. 101321, 2022, doi: 10.1016/j.csl.2021.101321.
T. Khan et al., "Assessing Parkinson's disease severity using speech analysis in non-native speakers,” Comput Speech Lang, vol. 61, 2020, doi: 10.1016/j.csl.2019.101047.
J. M. Tracy, Y. Özkanca, D. C. Atkins, and R. Hosseini Ghomi, "Investigating voice as a biomarker: Deep phenotyping methods for early detection of Parkinson's disease,” J Biomed Inform, vol. 104, no. December 2019, p. 103362, 2020, doi: 10.1016/j.jbi.2019.103362.
L. Ali, C. Chakraborty, Z. He, W. Cao, Y. Imrana, and J. J. P. C. Rodrigues, "A novel sample and feature dependent ensemble approach for Parkinson's disease detection,” Neural Computing and Applications, vol. 35, no. 22, pp. 15997–16010, 2022, doi: 10.1007/s00521-022-07046-2.
M. A. Muslim et al., "New model combination meta-learner to improve accuracy prediction P2P lending with stacking ensemble learning,” Intelligent Systems with Applications, vol. 18, no. December 2022, p. 200204, 2023, doi: 10.1016/j.iswa.2023.200204.
D. A. A. Pertiwi, P. R. Setyorini, M. A. Muslim, and E. Sugiharti, "Implementation of Discretisation and Correlation-based Feature Selection to Optimize Support Vector Machine in Diagnosis of Chronic Kidney Disease,” Buletin Ilmiah Sarjana Teknik Elektro, vol. 5, no. 2, pp. 201–209, 2023, doi: 10.12928/biste.v5i2.7548.
A. Falasari and M. A. Muslim, "Optimize Naí¯ve Bayes Classifier Using Chi Square and Term Frequency Inverse Document Frequency For Amazon Review Sentiment Analysis,” Journal of Soft Computing Exploration, vol. 3, no. 1, pp. 31–36, 2022, doi: 10.52465/joscex.v3i1.68.
R. Reddy and U. M. A. Kumar, "Classification of user's review using modified logistic regression technique,” International Journal of System Assurance Engineering and Management, 2022, doi: 10.1007/s13198-022-01711-4.
F. W. Nugraha, A. Harjoko, M. A. Muslim, and N. N. Alabid, "Mix histogram and gray level co-occurrence matrix to improve glaucoma prediction machine learning,” Journal of Soft Computing Exploration, vol. 4, no. 1, pp. 13–22, 2022, doi: 10.52465/joscex.v4i1.99.
C. Qin, B. Li, and B. Han, "Fast brain tumor detection using adaptive stochastic gradient descent on shared-memory parallel environment,” Engineering Applications of Artificial Intelligence, vol. 120, no. January, p. 105816, 2023, doi: 10.1016/j.engappai.2022.105816.
A. Biswas and M. S. Islam, "A Hybrid Deep CNN-SVM Approach for Brain Tumor Classification,” Journal of Information Systems Engineering and Business Intelligence, vol. 9, no. 1, pp. 0–15, 2023, doi: 10.20473/jisebi.9.1.1-15.
V. Vinay Kumar and P. Grace Kanmani Prince, "Deep belief network Assisted quadratic logit boost classifier for brain tumor detection using MR images,” Biomedical Signal Processing and Control, vol. 81, no. October 2022, p. 104415, 2023, doi: 10.1016/j.bspc.2022.104415.
J.-N. Eckardt et al., "Prediction of Complete Remission and Survival in Acute Myeloid Leukemia Using Supervised Machine Learning,” Blood, vol. 138, no. Supplement 1, pp. 108–108, 2021, doi: 10.1182/blood-2021-149582.
T. Mustaqim, C. Fatichah, and N. Suciati, "Combination of Cross Stage Partial Network and GhostNet with Spatial Pyramid Pooling on Yolov4 for Detection of Acute Lymphoblastic Leukemia Subtypes in Multi-Cell Blood Microscopic Image,” Scientific Journal of Informatics, vol. 9, no. 2, pp. 139–148, 2022, doi: 10.15294/sji.v9i2.37350.
S. Saravanan et al., "A Systematic Review of Artificial Intelligence (AI) Based Approaches for the Diagnosis of Parkinson's Disease,” Archives of Computational Methods in Engineering, vol. 29, no. 6, pp. 3639–3653, 2022, doi: 10.1007/s11831-022-09710-1.
L. Sigcha et al., "Deep learning and wearable sensors for the diagnosis and monitoring of Parkinson's disease: A systematic review,” Expert Syst Appl, vol. 229, no. PA, p. 120541, 2023, doi: 10.1016/j.eswa.2023.120541.
X. Tao, X. Guo, Y. Zheng, X. Zhang, and Z. Chen, "Self-adaptive oversampling method based on the complexity of minority data in imbalanced datasets classification,” Knowl Based Syst, vol. 277, p. 110795, 2023, doi: 10.1016/j.knosys.2023.110795.
A. Balakrishnan, J. Medikonda, P. K. Namboothiri, and M. Natarajan, "Mahalanobis Metric-based Oversampling Technique for Parkinson's Disease Severity Assessment using Spatiotemporal Gait Parameters,” Biomed Signal Process Control, vol. 86, no. PA, p. 105057, 2023, doi: 10.1016/j.bspc.2023.105057.
L. S. Lin, Y. S. Lin, D. C. Li, and Y. T. Chen, "Mega trend diffusion-siamese network oversampling for imbalanced datasets' SVM classification,” Appl Soft Comput, vol. 143, no. September 2022, p. 110406, 2023, doi: 10.1016/j.asoc.2023.110406.
A. Daza Vergaray, J. C. H. Miranda, J. B. Cornelio, A. R. López Carranza, and C. F. Ponce Sánchez, "Predicting the depression in university students using stacking ensemble techniques over oversampling method,” Inform Med Unlocked, vol. 41, no. June, p. 101295, 2023, doi: 10.1016/j.imu.2023.101295.
K. Zhu, M. Yin, D. Zhu, X. Zhang, C. Gao, and J. Jiang, "SCGRU: A general approach for identifying multiple classes of self-admitted technical debt with text generation oversampling,” Journal of Systems and Software, vol. 195, p. 111514, 2023, doi: 10.1016/j.jss.2022.111514.
A. F. Mulyana, W. Puspita, and J. Jumanto, "Increased accuracy in predicting student academic performance using random forest classifier,” Journal of Student Research Exploration, vol. 1, no. 2, pp. 94–103, Jul. 2023, doi: 10.52465/josre.v1i2.169.
T. L. Nikmah, N. Hazimah, S. Harahap, and G. C. Utami, "USING K-MEANS AND LRFM FEATURE SELECTION IN,” vol. 7, no. 1, pp. 21–28, 2023.
H. Lim, "Low-rank learning for feature selection in multi-label classification,” Pattern Recognit Lett, vol. 172, pp. 106–112, 2023, doi: 10.1016/j.patrec.2023.05.036.
T. Klonecki, P. Teisseyre, and J. Lee, "Cost-constrained feature selection in multilabel classification using an information-theoretic approach,” Pattern Recognit, vol. 141, p. 109605, 2023, doi: 10.1016/j.patcog.2023.109605.
Y. Ma, X. Yu, X. Luo, D. Liu, and Y. Zhang, "Adaptive feature selection for image steganalysis based on classification metrics,” Inf Sci (N Y), vol. 644, no. December 2022, p. 118973, 2023, doi: 10.1016/j.ins.2023.118973.
M. A. N. D. Sewwandi, Y. Li, and J. Zhang, "A class-specific feature selection and classification approach using neighborhood rough set and K-nearest neighbor theories,” Appl Soft Comput, vol. 143, p. 110366, 2023, doi: 10.1016/j.asoc.2023.110366.
W. F. Abror, A. Alamsyah, and M. Aziz, "Bankruptcy Prediction Using Genetic Algorithm-Support Vector Machine (GA-SVM) Feature Selection and Stacking,” Journal of Information System Exploration and Research, vol. 1, no. 2, Jul. 2023, doi: 10.52465/joiser.v1i2.180.
X. Yan and M. Jia, "A novel optimized SVM classification algorithm with multi-domain feature and its application to fault diagnosis of rolling bearing,” Neurocomputing, vol. 313, pp. 47–64, 2018, doi: 10.1016/j.neucom.2018.05.002.
B. Vidya and S. P, "Gait based Parkinson's disease diagnosis and severity rating using multi-class support vector machine,” Appl Soft Comput, vol. 113, p. 107939, 2021, doi: 10.1016/j.asoc.2021.107939.
A. U. Dullah, F. N. Apsari, and J. Jumanto, "Ensemble learning technique to improve breast cancer classification model,” Journal of Soft Computing Exploration, vol. 4, no. 2, Jun. 2023, doi: 10.52465/joscex.v4i2.166.
A. Singh, A. Jain, and S. E. Biable, "Financial Fraud Detection Approach Based on Firefly Optimization Algorithm and Support Vector Machine,” Applied Computational Intelligence and Soft Computing, vol. 2022, no. Cc, 2022, doi: 10.1155/2022/1468015.
Y. Dasril, M. A. Muslim, M. F. Al Hakim, J. Jumanto, and B. Prasetiyo, "Credit Risk Assessment in P2P Lending Using LightGBM and Particle Swarm Optimization,” Register: Jurnal Ilmiah Teknologi Sistem Informasi, vol. 9, no. 1, pp. 18–28, Feb. 2023, doi: 10.26594/register.v9i1.3060.
Y. P. Huang and M. F. Yen, "A new perspective of performance comparison among machine learning algorithms for financial distress prediction,” Applied Soft Computing Journal, vol. 83, p. 105663, 2019, doi: 10.1016/j.asoc.2019.105663.
A. Razzaque and D. A. Badholia, "PCA based feature extraction and MPSO based feature selection for gene expression microarray medical data classification,” Measurement: Sensors, vol. 31, no. January 2023, p. 100945, 2023, doi: 10.1016/j.measen.2023.100945.
D. GhoshRoy, P. A. Alvi, and K. C. Santosh, "Explainable AI to Predict Male Fertility Using Extreme Gradient Boosting Algorithm with SMOTE,” Electronics (Switzerland), vol. 12, no. 1, pp. 1–16, 2023, doi: 10.3390/electronics12010015.
Y. Li et al., "Multi-parameter methane measurement using near-infrared tunable diode laser absorption spectroscopy based on back propagation neural network,” Infrared Phys Technol, vol. 125, no. June, p. 104275, 2022, doi: 10.1016/j.infrared.2022.104275.
C. K. Jha and M. H. Kolekar, "Cardiac arrhythmia classification using tunable Q-wavelet transform based features and support vector machine classifier,” Biomed Signal Process Control, vol. 59, p. 101875, 2020, doi: 10.1016/j.bspc.2020.101875.
S. Mezzah and A. Tari, "Practical hyperparameters tuning of convolutional neural networks for EEG emotional features classification,” Intelligent Systems with Applications, vol. 18, no. January, p. 200212, 2023, doi: 10.1016/j.iswa.2023.200212.
S. Nematzadeh, F. Kiani, M. Torkamanian-Afshar, and N. Aydin, "Tuning hyperparameters of machine learning algorithms and deep neural networks using metaheuristics: A bioinformatics study on biomedical and biological cases,” Comput Biol Chem, vol. 97, no. December 2021, p. 107619, 2022, doi: 10.1016/j.compbiolchem.2021.107619.
R. G. Mantovani, A. L. D. Rossi, E. Alcobaça, J. Vanschoren, and A. C. P. L. F. de Carvalho, "A meta-learning recommender system for hyperparameter tuning: Predicting when tuning improves SVM classifiers,” Inf Sci (N Y), vol. 501, pp. 193–221, 2019, doi: 10.1016/j.ins.2019.06.005.
O. Yaman, F. Ertam, and T. Tuncer, "Automated Parkinson's disease recognition based on statistical pooling method using acoustic features,” Med Hypotheses, vol. 135, no. November 2019, p. 109483, 2020, doi: 10.1016/j.mehy.2019.109483.
P. Ghaheri, H. Nasiri, A. Shateri, and A. Homafar, "Diagnosis of Parkinson's Disease Based on Voice Signals Using SHAP and Hard Voting Ensemble Method,” pp. 1–19, 2022.
G. Solana-Lavalle and R. Rosas-Romero, "Analysis of voice as an assisting tool for detection of Parkinson's disease and its subsequent clinical interpretation,” Biomed Signal Process Control, vol. 66, p. 102415, 2021, doi: 10.1016/j.bspc.2021.102415.
J. S. Almeida et al., "Detecting Parkinson's disease with sustained phonation and speech signals using machine learning techniques,” Pattern Recognition Letters, vol. 125, pp. 55–62, 2019, doi: 10.1016/j.patrec.2019.04.005.
I. Ahmed, S. Aljahdali, M. S. Khan, and S. Kaddoura, "Classification of parkinson disease based on patient's voice signal using machine learning,” Intelligent Automation and Soft Computing, vol. 32, no. 2, pp. 705–722, 2022, doi: 10.32604/iasc.2022.022037.
F. Avantaggiato et al., "Neurobiology of Disease Intelligibility of speech in Parkinson ' s disease relies on anatomically segregated subthalamic beta oscillations,” Neurobiol Dis, vol. 185, no. July, p. 106239, 2023, doi: 10.1016/j.nbd.2023.106239.
D. Sztaho, I. Valalik, and K. Vicsi, "Parkinson's disease severity estimation on hungarian speech using various speech tasks,” 2019 10th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2019, 2019, doi: 10.1109/SPED.2019.8906277.
M. Wodzinski, A. Skalski, D. Hemmerling, J. R. Orozco-Arroyave, and E. Noth, "Deep Learning Approach to Parkinson's Disease Detection Using Voice Recordings and Convolutional Neural Network Dedicated to Image Classification,” Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, pp. 717–720, 2019, doi: 10.1109/EMBC.2019.8856972.
M. Nilashi, O. Ibrahim, S. Samad, H. Ahmadi, L. Shahmoradi, and E. Akbari, "An analytical method for measuring the Parkinson's disease progression: A case on a Parkinson's telemonitoring dataset,” Measurement (Lond), vol. 136, pp. 545–557, 2019, doi: 10.1016/j.measurement.2019.01.014.
"parkinsons.” [Online]. Available: https://www.openml.org/search?type=data&status=active&id=1488&sort=runs
Z. Xue, H. Lu, T. Zhang, J. Xu, and X. Guo, "A local dynamic feature selection fusion method for voice diagnosis of Parkinson's disease,” Comput Speech Lang, vol. 82, no. February, p. 101536, 2023, doi: 10.1016/j.csl.2023.101536.
S. Davoudi, A. Ahmadi, and M. R. Daliri, "Frequency–amplitude coupling: a new approach for decoding of attended features in covert visual attention task,” Neural Comput Appl, vol. 33, no. 8, pp. 3487–3502, 2021, doi: 10.1007/s00521-020-05222-w.
B. Prasetiyo, Alamsyah, M. F. Al Hakim, Jumanto, and M. H. Adi, "Differential augmentation data for vehicle classification using convolutional neural network,” 2023, p. 040001. doi: 10.1063/5.0126720.
G. Biagetti, P. Crippa, L. Falaschetti, S. Luzzi, and C. Turchetti, "Classification of Alzheimer's disease from EEG signal using robust-PCA feature extraction,” Procedia Comput Sci, vol. 192, no. 2019, pp. 3114–3122, 2021, doi: 10.1016/j.procs.2021.09.084.
J. Tao, Z. Yu, R. Zhang, and F. Gao, "RBF neural network modeling approach using PCA based LM–GA optimization for coke furnace system,” Appl Soft Comput, vol. 111, p. 107691, 2021, doi: 10.1016/j.asoc.2021.107691.
S. Gholinejad, A. Amiri-Simkooei, S. H. Alizadeh Moghaddam, and A. Alizadeh Naeini, "An automated PCA-based approach towards optimization of the rational function model,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 165, no. March, pp. 133–139, 2020, doi: 10.1016/j.isprsjprs.2020.05.011.
G. Celik and E. Başaran, "Proposing a new approach based on convolutional neural networks and random forest for the diagnosis of Parkinson's disease from speech signals,” Applied Acoustics, vol. 211, 2023, doi: 10.1016/j.apacoust.2023.109476.
R. Khaskhoussy and Y. Ben Ayed, "Improving Parkinson's disease recognition through voice analysis using deep learning,” Pattern Recognit Lett, vol. 168, pp. 64–70, 2023, doi: 10.1016/j.patrec.2023.03.011.
J. W. M. D. Souza, S. S. A. Alves, E. D. S. Rebouças, J. S. Almeida, and P. P. Rebouças Filho, "A New Approach to Diagnose Parkinson's Disease Using a Structural Cooccurrence Matrix for a Similarity Analysis,” Comput Intell Neurosci, vol. 2018, 2018, doi: 10.1155/2018/7613282.
R. G. Mantovani, A. L. D. Rossi, E. Alcobaça, J. Vanschoren, and A. C. P. L. F. de Carvalho, "A meta-learning recommender system for hyperparameter tuning: Predicting when tuning improves SVM classifiers,” Inf Sci (N Y), vol. 501, pp. 193–221, 2019, doi: 10.1016/j.ins.2019.06.005.
S. Aich, H. C. Kim, K. Younga, K. L. Hui, A. A. Al-Absi, and M. Sain, "A Supervised Machine Learning Approach using Different Feature Selection Techniques on Voice Datasets for Prediction of Parkinson's Disease,” International Conference on Advanced Communication Technology, ICACT, vol. 2019-Febru, no. 3, pp. 1116–1121, 2019, doi: 10.23919/ICACT.2019.8701961.
N. Peker and C. Kubat, "Application of Chi-square discretization algorithms to ensemble classification methods,” Expert Systems with Applications, vol. 185, no. July, p. 115540, 2021, doi: 10.1016/j.eswa.2021.115540.
E. Avuçlu and A. Elen, "Evaluation of train and test performance of machine learning algorithms and Parkinson diagnosis with statistical measurements,” Med Biol Eng Comput, vol. 58, no. 11, pp. 2775–2788, 2020, doi: 10.1007/s11517-020-02260-3.
R. Sheibani, E. Nikookar, and S. Alavi, "An ensemble method for diagnosis of Parkinson's disease based on voice measurements,” J Med Signals Sens, vol. 9, no. 4, pp. 221–226, 2019, doi: 10.4103/jmss.JMSS_57_18.
V. Despotovic, T. Skovranek, and C. Schommer, "Speech Based Estimation of Parkinson's Disease Using Gaussian Processes and Automatic Relevance Determination,” Neurocomputing, vol. 401, pp. 173–181, 2020, doi: 10.1016/j.neucom.2020.03.058.
Copyright (c) 2024 The Authors. Published by Universitas Airlangga.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
All accepted papers will be published under a Creative Commons Attribution 4.0 International (CC BY 4.0) License. Authors retain copyright and grant the journal right of first publication. CC-BY Licenced means lets others to Share (copy and redistribute the material in any medium or format) and Adapt (remix, transform, and build upon the material for any purpose, even commercially).
