Development and Usability Evaluation of an AI-Based Vocal Emotion Assessment Prototype for Vocational Communication Training
DOI:
https://doi.org/10.62238/chatra.v4i3.476Keywords:
artificial intelligence, prototype development, usability evaluation, vocational communication trainingAbstract
Vocational communication training requires opportunities for learners to practise vocal delivery and receive timely feedback, yet conventional assessment often depends on direct instructor observation. This study developed and evaluated a web-based AI-assisted vocal emotion assessment prototype for vocational communication practice. Using evolutionary prototyping, the system integrated browser-based audio recording and file upload, signal preprocessing, emotion2vec+ inference, probability visualization, and automated feedback. Functional performance was examined through eight black-box testing scenarios, while perceived usability was evaluated using the System Usability Scale with 15 participants comprising 12 vocational students and three instructors. All eight functional scenarios operated successfully, with client-server response times of approximately 0.45–0.65 seconds. The prototype achieved a SUS score of 76.98, indicating favorable perceived usability, particularly for ease of use and learnability. These findings support the prototype's preliminary feasibility as a formative practice tool, although classification validity and educational effectiveness require further evaluation using larger samples, expert-validated emotion labels, and authentic vocational communication settings.
Pelatihan komunikasi vokasional memerlukan kesempatan bagi peserta didik untuk mempraktikkan penyampaian vokal dan memperoleh umpan balik secara tepat waktu, sedangkan asesmen konvensional masih banyak bergantung pada pengamatan langsung instruktur. Penelitian ini mengembangkan dan mengevaluasi prototipe asesmen emosi vokal berbasis web dengan dukungan kecerdasan buatan untuk latihan komunikasi vokasional. Pengembangan menggunakan evolutionary prototyping dengan mengintegrasikan perekaman dan pengunggahan audio, prapemrosesan sinyal, inferensi emotion2vec+, visualisasi probabilitas, serta umpan balik otomatis. Kinerja fungsional diuji melalui delapan skenario black-box testing, sedangkan persepsi usability dievaluasi menggunakan System Usability Scale terhadap 15 partisipan yang terdiri atas 12 peserta didik vokasi dan tiga instruktur. Seluruh skenario pengujian fungsional berhasil dijalankan dengan waktu respons client-server sekitar 0,45–0,65 detik. Prototipe memperoleh skor SUS sebesar 76,98 yang menunjukkan persepsi usability yang positif, terutama pada kemudahan penggunaan dan pembelajaran sistem. Temuan ini mendukung kelayakan awal prototipe sebagai alat latihan formatif, tetapi validitas klasifikasi dan efektivitas pendidikan masih memerlukan pengujian lebih lanjut dengan sampel yang lebih luas, label emosi tervalidasi pakar, dan konteks komunikasi vokasional autentik.
References
Abbaschian, B. J., Sierra-Sosa, D., & Elmaghraby, A. (2021). Deep Learning Techniques for Speech Emotion Recognition, from Databases to Models. Sensors, 21(4), 1249. https://doi.org/10.3390/s21041249
Abraham, R. M., & Singaram, V. S. (2019). Using deliberate practice framework to assess the quality of feedback in undergraduate clinical skills training. BMC Medical Education, 19(1), 105. https://doi.org/10.1186/s12909-019-1547-5
Aftab, A., Morsali, A., Ghaemmaghami, S., & Champagne, B. (2022). LIGHT-SERNET: A lightweight fully convolutional neural network for speech emotion recognition. 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 6912–6916. https://doi.org/10.1109/ICASSP43922.2022.9746405
Alfredo, R., Echeverria, V., Jin, Y., Yan, L., Swiecki, Z., Gašević, D., & Martinez-Maldonado, R. (2024). Human-centred learning analytics and AI in education: A systematic literature review. Computers and Education: Artificial Intelligence, 6, 100215. https://doi.org/10.1016/j.caeai.2024.100215
Banihashem, S. K., Noroozi, O., van Ginkel, S., Macfadyen, L. P., & Biemans, H. J. A. (2022). A systematic review of the role of learning analytics in enhancing feedback practices in higher education. Educational Research Review, 37, 100489. https://doi.org/10.1016/j.edurev.2022.100489
Brooke, J. (1996). SUS: A “Quick and Dirty” Usability Scale. In P. W. Jordan, B. Thomas, I. L. McClelland, & B. Weerdmeester (Eds.), Usability Evaluation in Industry (pp. 189–194). Taylor & Francis. https://doi.org/10.1201/9781498710411-35
Budiman, R. D. A. (2025). Effectiveness of AI-Driven Assessments in Enhancing Learning Evaluation through Predictive Technology in Vocational Secondary School. International Journal of Information and Education Technology, 15(7), 1410–1417. https://doi.org/10.18178/ijiet.2025.15.7.2342
Camburn, B., Viswanathan, V., Linsey, J., Anderson, D., Jensen, D., Crawford, R., Otto, K., & Wood, K. (2017). Design prototyping methods: state of the art in strategies, techniques, and guidelines. Design Science, 3, e13. https://doi.org/DOI: 10.1017/dsj.2017.10
Cao, H., Cooper, D. G., Keutmann, M. K., Gur, R. C., Nenkova, A., & Verma, R. (2014). CREMA-D: Crowd-sourced emotional multimodal actors dataset. IEEE Transactions on Affective Computing, 5(4), 377–390. https://doi.org/10.1109/TAFFC.2014.2315692
Creswell, J. W., & Creswell, J. D. (2018). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage Publications.
De Vreugd, L., Van Leeuwen, A., Jansen, R., & Van der Schaaf, M. (2024). Learning Analytics Dashboard Design and Evaluation to Support Student Self-Regulation of Study Behaviour. Journal of Learning Analytics, 1–14. https://doi.org/10.18608/jla.2024.8529
El Ayadi, M., Kamel, M. S., & Karray, F. (2011). Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern Recognition, 44(3), 572–587. https://doi.org/10.1016/j.patcog.2010.09.020
Genito, P. (2026). The impact of AI on reading and literacy practices: implications for vocational training in the Adriatic-Ionian Region. Economia Marche-Journal of Applied Economics, 45(1).
George, M. J., & Odgers, C. L. (2015). Seven fears and the science of how mobile technologies may be influencing adolescents in the digital age. Perspectives on Psychological Science, 10(6), 832–851.
Haigler, K. (2021). The role of occupational research in complex assessments: Aligning educational practices with workplace requirements. The Journal of Writing Analytics, 5(1), 259–291.
Hyzy, M., Bond, R., Mulvenna, M., Bai, L., Dix, A., Leigh, S., & Hunt, S. (2022). System Usability Scale Benchmarking for Digital Health Apps: Meta-analysis. JMIR MHealth and UHealth, 10(8), e37290. https://doi.org/10.2196/37290
Kebede, A., Asgedom, A., & Asfaw, A. (2024). Is linking technical and vocational training with industry a bridge to employability? Lessons from a literature review for Ethiopia. Cogent Education, 11(1), 2406721.
Laguna, R. F. D., Aguinaga, D. L., Torres, D. E. C., & Cueto, B. A. L. (2025). Soft skills and the use of industry 4.0 as determinants of professional development in engineering graduates: A SEM approach. Sustainable Futures, 10, 100742.
Leem, S.-G., Fulford, D., Onnela, J.-P., Gard, D., & Busso, C. (2021). Separation of emotional and reconstruction embeddings on ladder network to improve speech emotion recognition robustness in noisy conditions. Proceedings of Interspeech 2021, 2871–2875. https://doi.org/10.21437/Interspeech.2021-1200
Lewis, J. R. (2018). The System Usability Scale: Past, Present, and Future. International Journal of Human-Computer Interaction, 34(7), 577–590. https://doi.org/10.1080/10447318.2018.1455307
Livingstone, S. R., & Russo, F. A. (2018). The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English. PLOS ONE, 13(5), e0196391. https://doi.org/10.1371/journal.pone.0196391
Ma, Z., Zheng, Z., Ye, J., Li, J., Gao, Z., Zhang, S., & Chen, X. (2024a). emotion2vec: Self-supervised pre-training for speech emotion representation. Findings of the Association for Computational Linguistics: ACL 2024, 15747–15760. https://doi.org/10.18653/v1/2024.findings-acl.933
Ma, Z., Zheng, Z., Ye, J., Li, J., Gao, Z., Zhang, S., & Chen, X. (2024b). emotion2vec: Self-supervised pre-training for speech emotion representation. Findings of the Association for Computational Linguistics: ACL 2024, 15747–15760. https://doi.org/10.18653/v1/2024.findings-acl.933
Matcha, W., Uzir, N. A., Gašević, D., & Pardo, A. (2020). A Systematic Review of Empirical Studies on Learning Analytics Dashboards: A Self-Regulated Learning Perspective. IEEE Transactions on Learning Technologies, 13(2), 226–245.
Munir, F. (2022). More than technical experts: Engineering professionals’ perspectives on the role of soft skills in their practice. Industry and Higher Education, 36(3), 294–305.
Ngo-Thi-Kim, H., Pham, Q. V, Phan, T. H. T., Dang, U. P., & Minh, B. N. H. (2026). Artificial intelligence and teacher education: A bibliometric analysis between 1991 and 2023. Journal of Educational and Social Research, 16(2), 631–642. https://doi.org/10.36941/jesr-2026-0314
Parra-Gallego, L. F., & Orozco-Arroyave, J. R. (2022). Classification of emotions and evaluation of customer satisfaction from speech in real world acoustic environments. Digital Signal Processing, 120, 103286. https://doi.org/10.1016/j.dsp.2021.103286
Paulsen, L., & Lindsay, E. (2024). Learning analytics dashboards are increasingly becoming about learning and not just analytics - A systematic review. Education and Information Technologies, 29(11), 14279–14308. https://doi.org/10.1007/s10639-023-12401-4
Picard, R. W. (1997). Affective computing. The MIT Press. https://doi.org/10.7551/mitpress/1140.001.0001
Qi, Z., Wang, J., Du, Y., & Zhang, X. (2026). The effect of blended learning approaches on vocational school students’ practical skills performance and learning behaviors: virtual simulations or hands-on activities while accessing instructional videos. Instructional Science, 54(1), 39.
Rivas, D. F., & Husein, S. (2022). Empathy, persuasiveness and knowledge promote innovative engineering and entrepreneurial skills. Education for Chemical Engineers, 40, 45–55.
Saputra, S. D., & Parhusip, J. (2026). PENGUKURAN USABILITY APLIKASI WEB MENGGUNAKAN SUS (SYSTEM USABILITY SCALE) DAN PENGUJIAN BLACK BOX PADA WEBSITE E-LEARNING (GEN-IT) KABUPATEN KATINGAN. Jurnal Informatika Dan Teknik Elektro Terapan, 14(1). https://doi.org/10.23960/jitet.v14i1.8390
Schuller, D., & Schuller, B. W. (2021). A review on five recent and near-future developments in computational processing of emotion in the human voice. Emotion Review, 13(1), 44–50. https://doi.org/10.1177/1754073919898526
Sidorova, M., Kopus, T., & Yurasova, I. (2024). Digital transformation for teaching management accounting: training with a simulation in an authentic professional environment. Accounting Education, 33(5), 621–648.
Suhirman, L., Nugroho, A. Y., Dhyanaputri, I. G. A. S., Adawiyah, R., Soraya, A., Dzaky, M. I., Alunaza, H., Akbar, S. A., Hafizi, M. Z., & Junida, D. S. (2026). Dasar Metodologi Penelitian. CV. Askara Sastra Media.
Tripathi, J. P., Bugalia, S., Burdak, K., & Abbas, S. (2021). Dynamical analysis and effects of law enforcement in a social interaction model. Physica A: Statistical Mechanics and Its Applications, 567, 125725. https://doi.org/10.1016/j.physa.2020.125725
Vakariuk, L. V. (2022). To the issue of the efficiency of legal regulation of labor relations and optimization of the labor-law regime. Naukovij Vìsnik Užgorodsʹkogo Nacìonalʹnogo Unìversitetu, 2(72), 9–14. https://doi.org/10.24144/2307-3322.2022.72.34
Waskito, Irzal, Wulansari, R. E., & Kyaw, Z. Y. (n.d.). The Adventure of Formative Assessment with Active Feedback in The Vocational Learning: The Empirical Effect for Increasing Students’ Achievement. Journal of Technical Education and Training, 14(1), 54–62. Retrieved https://journal.uthm.edu.my/index.php/JTET/article/view/9793
Zhang, S., Liu, R., Tao, X., & Zhao, X. (2021). Deep Cross-Corpus Speech Emotion Recognition: Recent Advances and Perspectives. Frontiers in Neurorobotics, 15, 784514. https://doi.org/10.3389/fnbot.2021.784514
Zhao, J., Mao, X., & Chen, L. (2019). Speech emotion recognition using deep 1D & 2D CNN LSTM networks. Biomedical Signal Processing and Control, 47, 312–323. https://doi.org/10.1016/j.bspc.2018.08.035
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Bayu Sulistiyanto Ipung Sutejo, Muhammad Revin Putra Fahlefi, Habibullah Akbar, Andriyanti Asianto, Diah Aryani, Ega Shela Marsiani (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
.png)
