Automated question generation and question answering from Turkish texts
Yıl: 2022 Cilt: 30 Sayı: 5 Sayfa Aralığı: 1931 - 1940 Metin Dili: İngilizce DOI: 10.55730/1300-0632.3914 İndeks Tarihi: 08-12-2022
Automated question generation and question answering from Turkish texts
Öz: While exam-style questions are a fundamental educational tool serving a variety of purposes, manual construction of questions is a complex process that requires training, experience and resources. Automatic question generation (QG) techniques can be utilized to satisfy the need for a continuous supply of new questions by streamlining their generation. However, compared to automatic question answering (QA), QG is a more challenging task. In this work, we fine-tune a multilingual T5 (mT5) transformer in a multitask setting for QA, QG and answer extraction tasks using Turkish QA datasets. To the best of our knowledge, this is the first academic work that performs automated text-to-text question generation from Turkish texts. Experimental evaluations show that the proposed multitask setting achieves state-of-the-art Turkish question answering and question generation performance on TQuADv1, TQuADv2 datasets and XQuAD Turkish split. The source code and the pretrained models are available at https://github.com/obss/turkish- question-generation.
Anahtar Kelime: Belge Türü: Makale Makale Türü: Araştırma Makalesi Erişim Türü: Erişime Açık
- [1] Kurdi G, Leo J, Parsia B, Sattler U, Al-Emari S. A systematic review of automatic question generation for educational purposes. International Journal of Artificial Intelligence in Education 2020; 30 (1): 121–204. doi:10.1007/s40593-019-00186-y.
- [2] Lee CH, Chen TY, Chen LP, Yang PC,Tsai RTH. Automatic question generation from children’s stories for companion chatbot. In: 2018 IEEE International Conference on Information Reuse and Integration (IRI); 2018. pp. 491–494.
- [3] Laban P, Canny J, Hearst MA. What’s the latest? a question-driven news chatbot. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations; Online; Association for Computational Linguistics; 2020. pp. 380–387.
- [4] Sreelakshmi A, Abhinaya S, Nair A, Nirmala SJ. A question answering and quiz generation chatbot for education. In: 2019 Grace Hopper Celebration India (GHCI); 2019. pp. 1–6.
- [5] Yue X, Zhang XF, Yao Z, Lin S, Sun H. CliniQG4QA: Generating diverse questions for domain adaptation of clinical question answering. In: 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM); 2021. pp. 580–587.
- [6] Mitkov R. Computer-aided generation of multiple-choice tests. In: HLT-NAACL 03 workshop on Building educa- tional applications using natural language processing; 2003. pp. 17–22.
- [7] Heilman M, Smith NA. Good question! statistical ranking for question generation. In: Human Language Technolo- gies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics; 2010. pp. 609–617.
- [8] Soleymanzadeh K. Domain specific automatic question generation from text. In: ACL 2017, Student Research Workshop; 2017. pp. 82–88.
- [9] Du X, Shao J, Cardie C. Learning to ask: Neural question generation for reading comprehension. In: 55th Annual Meeting of the Association for Computational Linguistics; 2017. pp. 1342–1352.
- [10] Hochreiter S, Schmidhuber J. Long short-term memory. Neural computation 1997; 9 (8): 1735–1780. doi: 10.1162/neco.1997.9.8.1735
- [11] Zhou Q, Yang N, Wei F, Tan C, Bao H et al. Neural question generation from text: A preliminary study. In: National CCF Conference on Natural Language Processing and Chinese Computing; 2017. pp. 662–671.
- [12] Zhao Y, Ni X, Ding Y, Ke Q. Paragraph-level neural question generation with maxout pointer and gated self- attention networks. In: 2018 Conference on Empirical Methods in Natural Language Processing; 2018. pp. 3901– 3910.
- [13] Duan N, Tang D, Chen P, Zhou M. Question generation for question answering. In: 2017 Conference on Empirical Methods in Natural Language Processing; 2017. pp. 866–874.
- [14] Kim Y, Lee H, Shin J, Jung K. Improving neural question generation using answer separation. In: AAAI Conference on Artificial Intelligence; 2019. pp. 6602–6609.
- [15] Sun X, Liu J, Lyu Y, He W, Ma Y et al. Answer-focused and position-aware neural question generation. In: 2018 Conference on Empirical Methods in Natural Language Processing; Brussels, Belgium; Association for Computa- tional Linguistics; 2018. pp. 3930–3939.
- [16] Tuan LA, Shah D, Barzilay R. Capturing greater context for question generation. In: AAAI Conference on Artificial Intelligence; 2020. pp. 9065–9072.
- [17] Kriangchaivech K ,Wangperawong A. Question generation by transformers. CoRR 2019; abs/1909.05017. doi: 10.48550/arXiv.1909.05017
- [18] Chan YH ,Fan YC. A recurrent BERT-based model for question generation. In: 2nd Workshop on Machine Reading for Question Answering; 2019. pp. 154–162.
- [19] Lopez LE, Cruz DK, Cruz JCB, Cheng C. Transformer-based end-to-end question generation. CoRR 2020; abs/2005.01107.
- [20] Xue L, Constant N, Roberts A, Kale M, Al-Rfou R et al. mt5: A massively multilingual pre-trained text-to-text transformer. CoRR 2020; abs/2010.11934. doi: 10.18653/v1/2021.naacl-main.41
- [21] Raffel C, Shazeer N, Roberts A, Lee K, Narang S et al. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research 2020; 21: 1–67. doi: 10.48550/arXiv.1910.10683
- [22] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN et al. Attention is all you need. Advances in neural information processing systems 2017; 30. doi: 10.48550/arXiv.1706.03762
- [23] Schweter S. BERTurk - BERT models for turkish 2020. doi: 10.5281/zenodo.3770924
- [24] Wolf T, Debut L, Sanh V, Chaumond J, Delangue C et al. Transformers: State-of-the-art natural language processing. In: 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations; 2020. pp. 38–45.
- [25] Paszke A, Gross S, Massa F, Lerer A, Bradbury J et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 2019; 32. doi: 10.48550/arXiv.1912.01703
- [26] Rajpurkar P, Zhang J, Lopyrev K, Liang P. Squad: 100, 000+ questions for machine comprehension of text. In: EMNLP; 2016.
- [27] Amidei J, Piwek P, Willis A. Evaluation methodologies in automatic question generation 2013-2018. In: 11th International Natural Language Generation Conference; 2018. pp. 307–317.
- [28] Papineni K, Roukos S, Ward T, Zhu WJ. BLEU: a method for automatic evaluation of machine translation. In: 40th annual meeting of the Association for Computational Linguistics; 2002. pp. 311–318.
- [29] Banerjee S,Lavie A. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In: ACL workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization; 2005. pp. 65–72.
- [30] Lin CY. ROUGE: A package for automatic evaluation of summaries. In: Text summarization branches out; 2004. pp. 74–81.
- [31] Soygazi F, Çiftçi O, Kök U, Cengiz S. THQuAD: Turkish historic question answering dataset for reading compre- hension. In: 2021 6th International Conference on Computer Science and Engineering (UBMK); 2021. pp. 215–220.
- [32] Artetxe M, Ruder S, Yogatama D. On the cross-lingual transferability of monolingual representations. CoRR 2019; abs/1910.11856. doi: 10.48550/arXiv.1910.11856
APA | Akyon F, Çavuşoğlu D, CENGİZ C, ALTINUÇ S, Temizel A (2022). Automated question generation and question answering from Turkish texts. , 1931 - 1940. 10.55730/1300-0632.3914 |
Chicago | Akyon Fatih Cagatay,Çavuşoğlu Devrim,CENGİZ CEMİL,ALTINUÇ Sinan Onur,Temizel Alptekin Automated question generation and question answering from Turkish texts. (2022): 1931 - 1940. 10.55730/1300-0632.3914 |
MLA | Akyon Fatih Cagatay,Çavuşoğlu Devrim,CENGİZ CEMİL,ALTINUÇ Sinan Onur,Temizel Alptekin Automated question generation and question answering from Turkish texts. , 2022, ss.1931 - 1940. 10.55730/1300-0632.3914 |
AMA | Akyon F,Çavuşoğlu D,CENGİZ C,ALTINUÇ S,Temizel A Automated question generation and question answering from Turkish texts. . 2022; 1931 - 1940. 10.55730/1300-0632.3914 |
Vancouver | Akyon F,Çavuşoğlu D,CENGİZ C,ALTINUÇ S,Temizel A Automated question generation and question answering from Turkish texts. . 2022; 1931 - 1940. 10.55730/1300-0632.3914 |
IEEE | Akyon F,Çavuşoğlu D,CENGİZ C,ALTINUÇ S,Temizel A "Automated question generation and question answering from Turkish texts." , ss.1931 - 1940, 2022. 10.55730/1300-0632.3914 |
ISNAD | Akyon, Fatih Cagatay vd. "Automated question generation and question answering from Turkish texts". (2022), 1931-1940. https://doi.org/10.55730/1300-0632.3914 |
APA | Akyon F, Çavuşoğlu D, CENGİZ C, ALTINUÇ S, Temizel A (2022). Automated question generation and question answering from Turkish texts. Turkish Journal of Electrical Engineering and Computer Sciences, 30(5), 1931 - 1940. 10.55730/1300-0632.3914 |
Chicago | Akyon Fatih Cagatay,Çavuşoğlu Devrim,CENGİZ CEMİL,ALTINUÇ Sinan Onur,Temizel Alptekin Automated question generation and question answering from Turkish texts. Turkish Journal of Electrical Engineering and Computer Sciences 30, no.5 (2022): 1931 - 1940. 10.55730/1300-0632.3914 |
MLA | Akyon Fatih Cagatay,Çavuşoğlu Devrim,CENGİZ CEMİL,ALTINUÇ Sinan Onur,Temizel Alptekin Automated question generation and question answering from Turkish texts. Turkish Journal of Electrical Engineering and Computer Sciences, vol.30, no.5, 2022, ss.1931 - 1940. 10.55730/1300-0632.3914 |
AMA | Akyon F,Çavuşoğlu D,CENGİZ C,ALTINUÇ S,Temizel A Automated question generation and question answering from Turkish texts. Turkish Journal of Electrical Engineering and Computer Sciences. 2022; 30(5): 1931 - 1940. 10.55730/1300-0632.3914 |
Vancouver | Akyon F,Çavuşoğlu D,CENGİZ C,ALTINUÇ S,Temizel A Automated question generation and question answering from Turkish texts. Turkish Journal of Electrical Engineering and Computer Sciences. 2022; 30(5): 1931 - 1940. 10.55730/1300-0632.3914 |
IEEE | Akyon F,Çavuşoğlu D,CENGİZ C,ALTINUÇ S,Temizel A "Automated question generation and question answering from Turkish texts." Turkish Journal of Electrical Engineering and Computer Sciences, 30, ss.1931 - 1940, 2022. 10.55730/1300-0632.3914 |
ISNAD | Akyon, Fatih Cagatay vd. "Automated question generation and question answering from Turkish texts". Turkish Journal of Electrical Engineering and Computer Sciences 30/5 (2022), 1931-1940. https://doi.org/10.55730/1300-0632.3914 |