An Efficient CNN Model for Automated Digital Handwritten Digit Classification
Downloads
Background: Handwriting recognition becomes an appreciable research area because of its important practical applications, but varieties of writing patterns make automatic classification a challenging task. Classifying handwritten digits with a higher accuracy is needed to improve the limitations from past research, which mostly used deep learning approaches.
Objective: Two most noteworthy limitations are low accuracy and slow computational speed. The current study is to model a Convolutional Neural Network (CNN), which is simple yet more accurate in classifying English handwritten digits for different datasets. Novelty of this paper is to explore an efficient CNN architecture that can classify digits of different datasets accurately.
Methods: The author proposed five different CNN architectures for training and validation tasks with two datasets. Dataset-1 consists of 12,000 MNIST data and Dataset-2 consists of 29,400-digit data of Kaggle. The proposed CNN models extract the features first and then performs the classification tasks. For the performance optimization, the models utilized stochastic gradient descent with momentum optimizer.
Results: Among the five models, one was found to be the best performer, with 99.53% and 98.93% of validation accuracy for Dataset-1 and Dataset-2 respectively. Compared to Adam and RMSProp optimizers, stochastic gradient descent with momentum yielded the highest accuracy.
Conclusion: The proposed best CNN model has the simplest architecture. It provides a higher accuracy for different datasets and takes less computational time. The validation accuracy of the proposed model is also higher than those of in past works.
Mahmoud M. Abu Ghosh, Ashraf Y. Maghari, "A Comparative Study on Handwriting Digit Recognition Using Neural Networks,” 2017 IEEE International Conference on Promising Electronic Technologies, pp. 77-81, 2017.
Kaensar C., "A Comparative Study on Handwriting Digit Recognition Classifier Using Neural Network, Support Vector Machine and K-Nearest Neighbor”, 9th International Conference on Computing and InformationTechnology (IC2IT2013), Advances in Intelligent Systems and Computing, Springer, vol 209, pp. 155-163.
Teddy Surya Gunawan, Ahmad Fakhrur Razi Mohd Noor, Mira Kartiwi, Development of English Handwritten Recognition Using Deep Neural Network, Indonesian Journal of Electrical Engineering and Computer Science, vol. 10, no. 2, , pp. 562-568, May 2018.
Saqib Ali, Zeeshan Shaukat, Muhammad Azeem, Zareen Sakhawat, Tariq Mahmood, Khalil ur Rehman, "An efcient and improved scheme for handwritten digit recognition based on convolutional neural network”, SN Applied Sciences, vol. 1, no. 9, August, 2019. DOI: https://doi.org/10.1007/s42452-019-1161-5
N. Shobha Rani, N. Chandan, A. Sajan Jain, H. R. Kiran, "Deformed character recognition using convolutional neural networks”, International Journal of Engineering & Technology, vol. 7, no. 3, pp. 1599-1604, 2018. DOI: 10.14419/ijet.v7i3.14053.
Adarsh Trivedi, Siddhant Srivastava, Apoorva Mishra, Anupam Shukla, Ritu Tiwari, "Hybrid evolutionary approach for Devanagari handwritten numeral recognition using Convolutional Neural Network”, The 6th International Conference on Smart Computing and Communications, Elsevier, vol. 125, pp. 525–532, 2018. DOI: 10.1016/j.procs.2017.12.068.
Nibaran Das, Ram Sarkar, Subhadip Basu, Mahantapas Kundu, Mita Nasipuri, Dipak Kumar Basu, "A genetic algorithm based region sampling for selection of local features in handwritten digit recognition application”, Applied Soft Computing, Elsevier, vol. 12, pp. 1592-1606., 2012. DOI: 10.1016/j.asoc.2011.11.030.
Ali A. Alani, "Arabic Handwritten Digit Recognition Based on Restricted Boltzmann Machine and Convolutional Neural Networks”, Multidisciplinary Digital Publishing Institute (MDPI), vol. 8, pp. 1-13, 2017. DOI: 10.3390/info8040142.
Ashadullah Shawon, Md. Jamil-Ur Rahman, Firoz Mahmud, M.M Arefin Zaman, "Bangla Handwritten Digit Recognition Using Deep CNN for Large and Unbiased Dataset”, International Conference on Bangla Speech and Language Processing (ICBSLP), IEEE, September 2018. DOI: 10.1109/ICBSLP.2018.8554900.
Durjoy Sen Maitra, Ujjwal Bhattacharya and Swapan K. Parui, "CNN Based Common Approach to Handwritten Character Recognition of Multiple Scripts”, 2015 13th International Conference on Document Analysis and Recognition (ICDAR), IEEE, pp. 1021-1025, August 2015. DOI: 10.1109/ICDAR.2015.7333916.
In-Jung Kim, Xiaohui Xie, "International Journal on Document Analysis and Recognition (IJDAR)”, International Journal on Document Analysis and Recognition (IJDAR), vol. 18, pp. 1-13, 2015. DOI: 10.1007/s10032-014-0229-4.
Hubert Cecotti, "Active graph based semi-supervised learning using image matching: application to handwritten digit recognition”, Pattern Recognition Letters, ELSEVIER, vol. 73, pp. 76-82, 1 April 2016. DOI: 10.1016/j.patrec.2016.01.016.
Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar and Pierre-Alain Muller, "Transfer learning for time series classification”, IEEE International Conference on Big Data 2018, pp. 1-11, November 2018. DOI: 10.1109/BigData.2018.8621990.
Xiao-Xiao Niu, Ching Y. Suen, "A novel hybrid CNN–SVM classifier for recognizing handwritten digits”, Pattern Recognition, ELSEVIER, vol. 45, pp. 3018-3025, 19 October 2011. DOI: 10.1016/j.patcog.2011.09.021.
Ranjan Jana and Siddhartha Bhattacharyya, "Character Recognition from Handwritten Image Using Convolutional Neural Networks”, Recent Trends in Signal and Image Processing, Advances in Intelligent Systems and Computing, Springer, Singapore, vol. 922, pp. 23-30, March 2019. DOI: 10.1007/978-981-13-6783-0_3.
Fathma Siddique, Shadman Sakib, Md. Abu Bakr Siddique, "Recognition of Handwritten Digit using Convolutional Neural Network in Python with Tensorflow and Comparison of Performance for Various Hidden Layers”, 5th International Conference on Advances in Electrical engineering (ICAEE), IEEE, pp. 541-546, September 2019. DOI: 10.20944/preprints201903.0039.v2.
Matthew Y.W. Teow, "Understanding Convolutional Neural Networks Using A Minimal Model for Handwritten Digit Recognition”, IEEE 2nd International Conference on Automatic Control and Intelligent Systems (I2CACIS 2017), pp. 167-172, October 2017. DOI: 10.1109/I2CACIS.2017.8239052.
Vladimir Golovko, Mikhno Egor, Aliaksandr Brich, and Anatoliy Sachenko, "A Shallow Convolutional Neural Network for Accurate Handwritten Digits Classification”, Communications in Computer and Information Science, Springer, vol. 673, pp. 77-85, February 2017. DOI: 10.1007/978-3-319-54220-1_8.
Dong-yuan Ge, Xi-fan Yao, Wen-jiang Xiang, Xue-jun Wen, En-chen Liu, "Design of High Accuracy Detector for MNIST Handwritten Digit Recognition Based on Convolutional Neural Network”, International Conference on Intelligent Computation Technology and Automation (ICICTA), IEEE, pp. 658-662, March 2020. DOI: 10.1109/ICICTA49267.2019.00145.
Saeed AL-Mansoori, "Intelligent Handwritten Digit Recognition using Artificial Neural Network”, Int. Journal of Engineering Research and Applications, vol. 5, no. 5, pp. 46-51, May 2015. DOI: 10.13140/RG.2.1.2466.0649.
Muthukrishnan Ramprasath, M.Vijay Anand, Shanmugasundaram Hariharan, "Image Classification using Convolutional Neural Networks”, International Journal of Pure and Applied Mathematics, vol. 119, no. 17, pp. 1307-1318, 2018.
Eva Tuba, Milan Tuba, Dana Simian, "Handwritten Digit Recognition by Support Vector Machine Optimized by Bat Algorithm”, WSCG 2016 - 24th Conference on Computer Graphics, Visualization and Computer Vision 2016, pp. 369-376, 2016.
Savita Ahlawata , Amit Choudhary, "Hybrid CNN-SVM Classifier for Handwritten Digit Recognition”, International Conference on Computational Intelligence and Data Science (ICCIDS 2019), Procedia Computer Science, Elsevier, Vol. 167, pp. 2554-2560, 2020. DOI: 10.1016/j.procs.2020.03.309.
Y. LeCun, Y. Bengio, and G. Hinton, "Deep Learning”, Nature, vol. 521, pp. 436-444, May 2015.
Rikiya Yamashita, Mizuho Nishio, Richard Kinh Gian Do & Kaori Togashi, "Convolutional neural networks: an overview and application in radiology”, Insights Imaging, Springer, vol. 9, pp. 611–629, June 2018. DOI: 10.1007/s13244-018-0639-9
Matthew Stewart, "Simple Introduction to Convolutional Neural Networks”, Towards Data Science, February 2019. Available at: https://towardsdatascience.com/simple-introduction-to-convolutional-neural-networks-cdf8d3077bac
Danqing Liu, "A Practical Guide to ReLU”, Medilum, November 2017. Available at: https://medium.com/@danqing/a-practical-guide-to-relu-b83ca804f1f7
Savita Ahlawat, Amit Choudhary, Anand Nayyar, Saurabh Singh and Byungun Yoon, "Improved Handwritten Digit Recognition Using Convolutional Neural Networks (CNN)”, Sensors, MDPI, vol. 20, pp. 1-18, June 2020. DOI: 10.3390/s20123344.
Saad ALBAWI , Tareq Abed MOHAMMED, Saad AL-ZAWI, "Understanding of a Convolutional Neural Network”, 2017 International Conference on Engineering and Technology (ICET), IEEE, March 2018. DOI: 10.1109/ICEngTechnol.2017.8308186.
Thomas Wood, "What is the Softmax Function”, Softmax Function, Deep AI, June 2019. Available at: https://deepai.org/machine-learning-glossary-and-terms/softmax-layer.
N. Kwak, "Introduction to Convolutional Neural Networks (CNNs),”2016. Available at: http://mipal.snu.ac.kr/images/5/5e/Intro2CNNs.pdf
"MNIST as .jpg”, Kaggle, 2016. Available at: https://www.kaggle.com/scolianni/mnistasjpg
"Handwritten Digit Classification”, Kaggle, 2020. Available at: https://www.kaggle.com/pradheeprio/handwritten-digit-classification
Diederik P. Kingma, Jimmy Lei Ba, "ADAM: A METHOD FOR STOCHASTIC OPTIMIZATION”, 3rd International Conference for Learning Representations, San Diego, pp. 1-15, 2015.
Thomas Kurbiel and Shahrzad Khaleghian, "Training of Deep Neural Networks based on Distance Measures using RMSProp”, Mathematics, Computer Science, ArXiv, 2017.
Ahmad Al-Hmouz, Ghazanfar Latif, Jaafar Alghazo, and Rami Al-Hmouz, "Enhanced Numeral Recognition for Handwritten Multi-language Numerals Using Fuzzy Set-Based Decision Mechanism”, International Journal of Machine Learning and Computing, vol. 10, no. 1, pp. 99-107, January 2020. DOI: 10.18178/ijmlc.2020.10.1.905
B. El Kessab, C. Daoui, B. Bouikhalene, M. Fakir, K. Moro, "Extraction Method of Handwritten Digit Recognition Tested on the MNIST Database”, International Journal of Advanced Science and Technology, vol. 50, pp. 99-110, January 2013.
Chayaporn Kaensar, "A Comparative Study on Handwriting Digit Recognition Classifier Using Neural Network, Support Vector Machine and K-Nearest Neighbor”, 9th International Conference on Computing and InformationTechnology, AISC 209, pp. 155-163, January 2013. DOI: 10.1007/978-3-642-37371-8_19
Ebrahimzadeh, Mahdi Jampour, "Efficient Handwritten Digit Recognition based on Histogram of Oriented Gradients and SVM", International Journal of Computer Applications, vol. 104, no.9, pp. 10-13, October 2014.
Mahmoud M. Abu Ghosh, Ashraf Y. Maghari, "A Comparative Study on Handwriting Digit Recognition Using Neural Networks”, 2017 International Conference on Promising Electronic Technologies, IEEE, pp.77-81, 2017. DOI: 10.1109/ICPET.2017.20
Shan Pang and Xinyi Yang, "Deep Convolutional Extreme Learning Machine and Its Application in Handwritten Digit Classification”, Computational Intelligence and Neuroscience, Hindawi, vol. 3, pp. 1-10, August 2016. DOI: 10.1155/2016/3049632
Hubert Cecotti, "Active graph based semi-supervised learning using image matching: application to handwritten digit recognition”, Pattern Recognition Letters, vol. 73, pp. 76-82, February 2016.DOI: 10.1016/j.patrec.2016.01.016
Bouchra EL QACIMY, Mounir AIT KERROUM and Ahmed HAMMOUCH, "Feature Extraction based on DCT for Handwritten Digit Recognition”, IJCSI International Journal of Computer Science Issues, vol. 11, Issue 6, no. 2, pp. 27-33, November 2014.
Andr´es Calder´on, Sergio Roa and Jorge Victorino, "Handwritten Digit Recognition using Convolutional Neural Networks and Gabor filters”, e International Congress on Computational Intelligence CIIC, 2003.
Hamayun A. Khan, "MCS HOG Features and SVM Based Handwritten Digit Recognition System”, Journal of Intelligent Learning Systems and Applications, vol. 09, no. 02, pp. 21-33, January 2017. DOI: 10.4236/jilsa.2017.92003
Mohd Razif Shamsuddin, Shuzlina Abdul-Rahman and Azlinah Mohamed, "Exploratory Analysis of MNIST Handwritten Digit for Machine Learning Modelling”, 4th International Conference 2018, Soft Computing in Data Science (SCDS),pp. 134-145, 2018.
Authors who publish with this journal agree to the following terms:
All accepted papers will be published under a Creative Commons Attribution 4.0 International (CC BY 4.0) License. Authors retain copyright and grant the journal right of first publication. CC-BY Licenced means lets others to Share (copy and redistribute the material in any medium or format) and Adapt (remix, transform, and build upon the material for any purpose, even commercially).
