Fatigue Detection on Face Image Using FaceNet Algorithm and K-Nearest Neighbor Classifier
Downloads
Background: The COVID-19 pandemic has made people spend more time on online meetings more than ever. The prolonged time looking at the monitor may cause fatigue, which can subsequently impact the mental and physical health. A fatigue detection system is needed to monitor the Internet users well-being. Previous research related to the fatigue detection system used a fuzzy system, but the accuracy was below 85%. In this research, machine learning is used to improve accuracy.
Objective: This research examines the combination of the FaceNet algorithm with either k-nearest neighbor (K-NN) or multiclass support vector machine (SVM) to improve the accuracy.
Methods: In this study, we used the UTA-RLDD dataset. The features used for fatigue detection come from the face, so the dataset is segmented using the Haar Cascades method, which is then resized. The feature extraction process uses FaceNet's pre-trained algorithm. The extracted features are classified into three classes”focused, unfocused, and fatigue”using the K-NN or multiclass SVM method.
Results: The combination between the FaceNet algorithm and K-NN, with a value of resulted in a better accuracy than the FaceNet algorithm with multiclass SVM with the polynomial kernel (at 94.68% and 89.87% respectively). The processing speed of both combinations of methods has allowed for real-time data processing.
Conclusion: This research provides an overview of methods for early fatigue detection while working at the computer so that we can limit staring at the computer screen too long and switch places to maintain the health of our eyes.
T. L. Wang and D. A. Vella-Brodrick, "Examining Screen Time, Screen Use Experiences, and Well-Being in Adults,” Soc. Netw., vol. 07, no. 01, pp. 32–44, 2018, doi: 10.4236/sn.2018.71003.
E. Neophytou, L. A. Manwell, and R. Eikelboom, "Effects of Excessive Screen Time on Neurodevelopment, Learning, Memory, Mental Health, and Neurodegeneration: a Scoping Review,” Int. J. Ment. Health Addict., no. December, 2019, doi: 10.1007/s11469-019-00182-2.f
S. Poudel, "A research report about effect of display gadgets on eyesight quality (computer vision syndrome) Of M.Sc.(CSIT) students in Tribhuvan University,” Int. J. Sci. Eng. Res., vol. 9, no. 8, 2018, doi: 10.14299/ijser.2018.99.
Z. Yan, L. Hu, H. Chen, and F. Lu, "Computer Vision Syndrome: A widely spreading but largely unknown epidemic among computer users,” Comput. Human Behav., vol. 24, no. 5, pp. 2026–2042, 2008, doi: 10.1016/j.chb.2007.09.004.
S. H. Meng, S. B. Hu, A. C. Huang, T. J. Huang, Z. Xie, and C. Jian, "Research on Eye Detection and Fatigue Early Warning Technologies,” in Advances in Intelligent Systems and Computing, 2018, vol. 682, pp. 3–9, doi: 10.1007/978-3-319-68527-4_1.
M. Toniolo-Barrios and L. Pitt, "Mindfulness and the challenges of working from home in times of crisis,” Bus. Horiz., no. January, pp. 1–19, 2020.
W. Shi, J. Li, and Y. Yang, "Face Fatigue Detection Method Based on MTCNN and Machine Vision,” in International Conference on Applications and Techniques in Cyber Intelligence ATCI 2019, 2020, pp. 233–240.
S. Palaniswamy and S. Tripathi, "Emotion recognition from facial expressions using images with pose, illumination and age variation for human-computer/robot interaction,” J. ICT Res. Appl., vol. 12, no. 1, pp. 14–34, 2018, doi: 10.5614/itbj.ict.res.appl.2018.12.1.2.
N. Zhang, H. Zhang, and J. Huang, "Driver fatigue state detection based on facial key points,” 2019 6th Int. Conf. Syst. Informatics, ICSAI 2019, no. Icsai, pp. 144–149, 2019, doi: 10.1109/ICSAI48974.2019.9010478.
H. Tao, G. Zhang, Y. Zhao, and Y. Zhou, "Real-time driver fatigue detection based on face alignment,” Ninth Int. Conf. Digit. Image Process. (ICDIP 2017), vol. 10420, no. Icdip, p. 1042003, 2017, doi: 10.1117/12.2282043.
T. Qunzhu, Z. Rui, Y. Yufei, Z. Chengyao, and L. Zhijun, "Improvement of random forest cascade regression algorithm and its application in fatigue detection,” 2019 2nd Int. Conf. Electron. Technol. ICET 2019, pp. 499–503, 2019, doi: 10.1109/ELTECH.2019.8839317.
Z. Li and L. Nianqiang, "Fatigue driving detection system based on face feature,” 2019 2nd Int. Conf. Electron. Technol. ICET 2019, pp. 525–529, 2019, doi: 10.1109/ELTECH.2019.8839479.
Y. Kong and W. Li, "Research on recognition method of learning concentration based on face feature,” 2017 IEEE Int. Conf. Cybern. Intell. Syst. CIS 2017 IEEE Conf. Robot. Autom. Mechatronics, RAM 2017 - Proc., vol. 2018-Janua, pp. 334–338, 2017, doi: 10.1109/ICCIS.2017.8274797.
F. Cahyono, W. Wirawan, and R. Fuad Rachmadi, "Face Recognition System using Facenet Algorithm for Employee Presence,” pp. 57–62, 2020, doi: 10.1109/icovet50258.2020.9229888.
T. Nyein and A. N. Oo, "University Classroom Attendance System Using FaceNet and Support Vector Machine,” in 2019 International Conference on Advanced Information Technologies (ICAIT), Nov. 2019, pp. 171–176, doi: 10.1109/AITC.2019.8921316.
F. Schroff, D. Kalenichenko, and J. Philbin, "FaceNet: A unified embedding for face recognition and clustering,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015, pp. 815–823, doi: 10.1109/CVPR.2015.7298682.
A. Swaminathan, M. Chaba, D. K. Sharma, and Y. Chaba, "Gender Classification using Facial Embeddings: A Novel Approach,” Procedia Comput. Sci., vol. 167, no. 2019, pp. 2634–2642, 2020, doi: 10.1016/j.procs.2020.03.342.
T. T. N. Thi and K. N. Trong, "An Efficient Face Detection and Recognition,” Int. J. Innov. Technol. Explor. Eng., vol. 7, no. 5, pp. 35–39, 2018.
C. Guo and Y. Yang, "Implementation of a specified face recognition system based on video,” Proc. 2019 IEEE 4th Adv. Inf. Technol. Electron. Autom. Control Conf. IAEAC 2019, no. Iaeac, pp. 79–84, 2019, doi: 10.1109/IAEAC47372.2019.8997627.
P. Viola and M. J. Jones, "Robust Real-Time Face Detection,” Int. J. Comput. Vis., vol. 57, no. 2, pp. 137–154, 2004, doi: 10.1023/B:VISI.0000013087.49260.fb.
R. Ghoddoosian, M. Galib, and V. Athitsos, "A realistic dataset and baseline temporal model for early drowsiness detection,” IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Work., vol. 2019-June, pp. 178–187, 2019, doi: 10.1109/CVPRW.2019.00027.
P. Viola and M. Jones, "Rapid Object Detection using a Boosted Cascade of Simple Features,” in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, 2001, pp. 511–518.
A. Susanto, D. Sinaga, C. A. Sari, E. H. Rachmawanto, and D. R. I. M. Setiadi, "A High Performance of Local Binary Pattern on Classify Javanese Character Classification,” Sci. J. Informatics, vol. 5, no. 1, p. 8, 2018, doi: 10.15294/sji.v5i1.14017.
V. Ong and D. Suhartono, "Using K-Nearest Neighbor in Optical Character Recognition,” ComTech Comput. Math. Eng. Appl., vol. 7, no. 1, p. 53, 2016, doi: 10.21512/comtech.v7i1.2223.
K. B. Duan, J. C. Rajapakse, and M. N. Nguyen, "One-versus-one and one-versus-all multiclass SVM-RFE for gene selection in cancer classification,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 4447 LNCS, pp. 47–56, 2007, doi: 10.1007/978-3-540-71783-6_5.
F. D. Adhinata, A. Harjoko, and Wahyono, "Object Searching on Video Using ORB Descriptor and Support Vector Machine,” in Advances in Computational Collective Intelligence, 2020, pp. 239–251.
Authors who publish with this journal agree to the following terms:
All accepted papers will be published under a Creative Commons Attribution 4.0 International (CC BY 4.0) License. Authors retain copyright and grant the journal right of first publication. CC-BY Licenced means lets others to Share (copy and redistribute the material in any medium or format) and Adapt (remix, transform, and build upon the material for any purpose, even commercially).
