Yıl: 2022 Cilt: 30 Sayı: 5 Sayfa Aralığı: 1931 - 1940 Metin Dili: İngilizce DOI: 10.55730/1300-0632.3914 İndeks Tarihi: 08-12-2022

Automated question generation and question answering from Turkish texts

Öz:
While exam-style questions are a fundamental educational tool serving a variety of purposes, manual construction of questions is a complex process that requires training, experience and resources. Automatic question generation (QG) techniques can be utilized to satisfy the need for a continuous supply of new questions by streamlining their generation. However, compared to automatic question answering (QA), QG is a more challenging task. In this work, we fine-tune a multilingual T5 (mT5) transformer in a multitask setting for QA, QG and answer extraction tasks using Turkish QA datasets. To the best of our knowledge, this is the first academic work that performs automated text-to-text question generation from Turkish texts. Experimental evaluations show that the proposed multitask setting achieves state-of-the-art Turkish question answering and question generation performance on TQuADv1, TQuADv2 datasets and XQuAD Turkish split. The source code and the pretrained models are available at https://github.com/obss/turkish- question-generation.
Anahtar Kelime: Turkish question answering question generation answer extraction multitask transformer

Belge Türü: Makale Makale Türü: Araştırma Makalesi Erişim Türü: Erişime Açık
  • [1] Kurdi G, Leo J, Parsia B, Sattler U, Al-Emari S. A systematic review of automatic question generation for educational purposes. International Journal of Artificial Intelligence in Education 2020; 30 (1): 121–204. doi:10.1007/s40593-019-00186-y.
  • [2] Lee CH, Chen TY, Chen LP, Yang PC,Tsai RTH. Automatic question generation from children’s stories for companion chatbot. In: 2018 IEEE International Conference on Information Reuse and Integration (IRI); 2018. pp. 491–494.
  • [3] Laban P, Canny J, Hearst MA. What’s the latest? a question-driven news chatbot. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations; Online; Association for Computational Linguistics; 2020. pp. 380–387.
  • [4] Sreelakshmi A, Abhinaya S, Nair A, Nirmala SJ. A question answering and quiz generation chatbot for education. In: 2019 Grace Hopper Celebration India (GHCI); 2019. pp. 1–6.
  • [5] Yue X, Zhang XF, Yao Z, Lin S, Sun H. CliniQG4QA: Generating diverse questions for domain adaptation of clinical question answering. In: 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM); 2021. pp. 580–587.
  • [6] Mitkov R. Computer-aided generation of multiple-choice tests. In: HLT-NAACL 03 workshop on Building educa- tional applications using natural language processing; 2003. pp. 17–22.
  • [7] Heilman M, Smith NA. Good question! statistical ranking for question generation. In: Human Language Technolo- gies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics; 2010. pp. 609–617.
  • [8] Soleymanzadeh K. Domain specific automatic question generation from text. In: ACL 2017, Student Research Workshop; 2017. pp. 82–88.
  • [9] Du X, Shao J, Cardie C. Learning to ask: Neural question generation for reading comprehension. In: 55th Annual Meeting of the Association for Computational Linguistics; 2017. pp. 1342–1352.
  • [10] Hochreiter S, Schmidhuber J. Long short-term memory. Neural computation 1997; 9 (8): 1735–1780. doi: 10.1162/neco.1997.9.8.1735
  • [11] Zhou Q, Yang N, Wei F, Tan C, Bao H et al. Neural question generation from text: A preliminary study. In: National CCF Conference on Natural Language Processing and Chinese Computing; 2017. pp. 662–671.
  • [12] Zhao Y, Ni X, Ding Y, Ke Q. Paragraph-level neural question generation with maxout pointer and gated self- attention networks. In: 2018 Conference on Empirical Methods in Natural Language Processing; 2018. pp. 3901– 3910.
  • [13] Duan N, Tang D, Chen P, Zhou M. Question generation for question answering. In: 2017 Conference on Empirical Methods in Natural Language Processing; 2017. pp. 866–874.
  • [14] Kim Y, Lee H, Shin J, Jung K. Improving neural question generation using answer separation. In: AAAI Conference on Artificial Intelligence; 2019. pp. 6602–6609.
  • [15] Sun X, Liu J, Lyu Y, He W, Ma Y et al. Answer-focused and position-aware neural question generation. In: 2018 Conference on Empirical Methods in Natural Language Processing; Brussels, Belgium; Association for Computa- tional Linguistics; 2018. pp. 3930–3939.
  • [16] Tuan LA, Shah D, Barzilay R. Capturing greater context for question generation. In: AAAI Conference on Artificial Intelligence; 2020. pp. 9065–9072.
  • [17] Kriangchaivech K ,Wangperawong A. Question generation by transformers. CoRR 2019; abs/1909.05017. doi: 10.48550/arXiv.1909.05017
  • [18] Chan YH ,Fan YC. A recurrent BERT-based model for question generation. In: 2nd Workshop on Machine Reading for Question Answering; 2019. pp. 154–162.
  • [19] Lopez LE, Cruz DK, Cruz JCB, Cheng C. Transformer-based end-to-end question generation. CoRR 2020; abs/2005.01107.
  • [20] Xue L, Constant N, Roberts A, Kale M, Al-Rfou R et al. mt5: A massively multilingual pre-trained text-to-text transformer. CoRR 2020; abs/2010.11934. doi: 10.18653/v1/2021.naacl-main.41
  • [21] Raffel C, Shazeer N, Roberts A, Lee K, Narang S et al. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research 2020; 21: 1–67. doi: 10.48550/arXiv.1910.10683
  • [22] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN et al. Attention is all you need. Advances in neural information processing systems 2017; 30. doi: 10.48550/arXiv.1706.03762
  • [23] Schweter S. BERTurk - BERT models for turkish 2020. doi: 10.5281/zenodo.3770924
  • [24] Wolf T, Debut L, Sanh V, Chaumond J, Delangue C et al. Transformers: State-of-the-art natural language processing. In: 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations; 2020. pp. 38–45.
  • [25] Paszke A, Gross S, Massa F, Lerer A, Bradbury J et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 2019; 32. doi: 10.48550/arXiv.1912.01703
  • [26] Rajpurkar P, Zhang J, Lopyrev K, Liang P. Squad: 100, 000+ questions for machine comprehension of text. In: EMNLP; 2016.
  • [27] Amidei J, Piwek P, Willis A. Evaluation methodologies in automatic question generation 2013-2018. In: 11th International Natural Language Generation Conference; 2018. pp. 307–317.
  • [28] Papineni K, Roukos S, Ward T, Zhu WJ. BLEU: a method for automatic evaluation of machine translation. In: 40th annual meeting of the Association for Computational Linguistics; 2002. pp. 311–318.
  • [29] Banerjee S,Lavie A. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In: ACL workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization; 2005. pp. 65–72.
  • [30] Lin CY. ROUGE: A package for automatic evaluation of summaries. In: Text summarization branches out; 2004. pp. 74–81.
  • [31] Soygazi F, Çiftçi O, Kök U, Cengiz S. THQuAD: Turkish historic question answering dataset for reading compre- hension. In: 2021 6th International Conference on Computer Science and Engineering (UBMK); 2021. pp. 215–220.
  • [32] Artetxe M, Ruder S, Yogatama D. On the cross-lingual transferability of monolingual representations. CoRR 2019; abs/1910.11856. doi: 10.48550/arXiv.1910.11856
APA Akyon F, Çavuşoğlu D, CENGİZ C, ALTINUÇ S, Temizel A (2022). Automated question generation and question answering from Turkish texts. , 1931 - 1940. 10.55730/1300-0632.3914
Chicago Akyon Fatih Cagatay,Çavuşoğlu Devrim,CENGİZ CEMİL,ALTINUÇ Sinan Onur,Temizel Alptekin Automated question generation and question answering from Turkish texts. (2022): 1931 - 1940. 10.55730/1300-0632.3914
MLA Akyon Fatih Cagatay,Çavuşoğlu Devrim,CENGİZ CEMİL,ALTINUÇ Sinan Onur,Temizel Alptekin Automated question generation and question answering from Turkish texts. , 2022, ss.1931 - 1940. 10.55730/1300-0632.3914
AMA Akyon F,Çavuşoğlu D,CENGİZ C,ALTINUÇ S,Temizel A Automated question generation and question answering from Turkish texts. . 2022; 1931 - 1940. 10.55730/1300-0632.3914
Vancouver Akyon F,Çavuşoğlu D,CENGİZ C,ALTINUÇ S,Temizel A Automated question generation and question answering from Turkish texts. . 2022; 1931 - 1940. 10.55730/1300-0632.3914
IEEE Akyon F,Çavuşoğlu D,CENGİZ C,ALTINUÇ S,Temizel A "Automated question generation and question answering from Turkish texts." , ss.1931 - 1940, 2022. 10.55730/1300-0632.3914
ISNAD Akyon, Fatih Cagatay vd. "Automated question generation and question answering from Turkish texts". (2022), 1931-1940. https://doi.org/10.55730/1300-0632.3914
APA Akyon F, Çavuşoğlu D, CENGİZ C, ALTINUÇ S, Temizel A (2022). Automated question generation and question answering from Turkish texts. Turkish Journal of Electrical Engineering and Computer Sciences, 30(5), 1931 - 1940. 10.55730/1300-0632.3914
Chicago Akyon Fatih Cagatay,Çavuşoğlu Devrim,CENGİZ CEMİL,ALTINUÇ Sinan Onur,Temizel Alptekin Automated question generation and question answering from Turkish texts. Turkish Journal of Electrical Engineering and Computer Sciences 30, no.5 (2022): 1931 - 1940. 10.55730/1300-0632.3914
MLA Akyon Fatih Cagatay,Çavuşoğlu Devrim,CENGİZ CEMİL,ALTINUÇ Sinan Onur,Temizel Alptekin Automated question generation and question answering from Turkish texts. Turkish Journal of Electrical Engineering and Computer Sciences, vol.30, no.5, 2022, ss.1931 - 1940. 10.55730/1300-0632.3914
AMA Akyon F,Çavuşoğlu D,CENGİZ C,ALTINUÇ S,Temizel A Automated question generation and question answering from Turkish texts. Turkish Journal of Electrical Engineering and Computer Sciences. 2022; 30(5): 1931 - 1940. 10.55730/1300-0632.3914
Vancouver Akyon F,Çavuşoğlu D,CENGİZ C,ALTINUÇ S,Temizel A Automated question generation and question answering from Turkish texts. Turkish Journal of Electrical Engineering and Computer Sciences. 2022; 30(5): 1931 - 1940. 10.55730/1300-0632.3914
IEEE Akyon F,Çavuşoğlu D,CENGİZ C,ALTINUÇ S,Temizel A "Automated question generation and question answering from Turkish texts." Turkish Journal of Electrical Engineering and Computer Sciences, 30, ss.1931 - 1940, 2022. 10.55730/1300-0632.3914
ISNAD Akyon, Fatih Cagatay vd. "Automated question generation and question answering from Turkish texts". Turkish Journal of Electrical Engineering and Computer Sciences 30/5 (2022), 1931-1940. https://doi.org/10.55730/1300-0632.3914