Improvement of Design of Multiple Choice Questions in Annual Residency Exams by Giving Feedback


Mahdi Beiglarkhani 1 , * , Azam Meyari 1

1 Iran

How to Cite: Beiglarkhani M , Meyari A. Improvement of Design of Multiple Choice Questions in Annual Residency Exams by Giving Feedback, Strides Dev Med Educ. 2013 ; 10(1):e60923.


Strides in Development of Medical Education: 10 (1); e60923
Published Online: May 15, 2013
Article Type: Research Article
Received: August 27, 2017
Accepted: March 20, 2013




Background & Objective: The aim of this study was to survey the effect of presenting feedback on improvement of multiple choice questions (MCQ) design in annual residency exams. Methods: In this semi-experimental study, the structure and taxonomy in MCQs of annual residency exams in 2009 were analyzed through Millman’s checklist. Based on this data, an interventional feedback on strengths and weaknesses was presented to question designers who had experience of attending MCQ workshops and designing questions in the past. The principles of giving technical feedback in medical education such as objectivity, right timing, and being direct, respectful and without judgment were considered. After the annual residency exam in 2010, the questions of designers in both years were compared in respect to Millman’s principals and taxonomy before and after the intervention. Results: From 2500 questions, 2035 questions were entered into the study. 65.2% of the questions developed in 2009 had no structural flaw; this increased to 82.8% after the intervention, which was a significant difference (P < 0.001). Moreover, there was a significant increase in the percentage of questions with a high taxonomy; from 38% in 2009 to 53.1% in 2010 (P < 0.001). Conclusion: Giving interventional feedback, based on its techniques in medical education, to experienced designers of MCQs can affect the improvement of designing quality of questions.


Multiple choice questions Residency Medical education Taxonomy Millman's principles Feedback

© 2013, Medical Education Development Center. This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License ( which permits copy and redistribute the material just in noncommercial usages, provided the original work is properly cited.


Fulltext is available in pdf


  • 1.

    Reference is available in pdf