Controlling item difficulty for automatic vocabulary question generation

Main Article Content

Yuni Susanti
Takenobu Tokunaga
Hitoshi Nishikawa
Hiroyuki Obari


The present study investigates the best factor for controlling the item difficulty of multiple-choice English vocabulary questions generated by an automatic question generation system. Three factors are considered for controlling item difficulty: (1) reading passage difficulty, (2) semantic similarity between the correct answer and distractors, and (3) the distractor word difficulty level. An experiment was conducted by administering machine-generated items to three groups of English learners. The groups were determined based on their standardised English test scores. In total, 120 items, generated using combinations of the above three factors, were tested. The results reveal that the distractor word difficulty level had the greatest impact on item difficulty, but this tendency changed depending on the proficiency of the test takers. These results will be of use when implementing a fully automatic system for administrating tests.


Metrics Loading ...

Article Details

How to Cite
Susanti, Y. ., Tokunaga, T. ., Nishikawa, H. ., & Obari, H. . (2017). Controlling item difficulty for automatic vocabulary question generation. Research and Practice in Technology Enhanced Learning, 12. Retrieved from