Design, Implementation, and Evaluation of a Medical Education Fellowship Program for the Faculty Members of Kerman University of Medical Sciences Based on the Kirkpatrick Model

AUTHORS

Mahmoud Reza Dehghani 1 , Mahla Salajegheh 2 , 3 , * , Majid Fasihi Harandi 4 , Kambiz Bahaadinbeigy 5 , Bahareh Bahman Bijari 6 , Zeynab Shakiba 7 , Zahra Fatahi 1

1 Medical Education Development Center, Kerman University of Medical Sciences, Kerman, Iran

2 Medical Education Department, Kerman University of Medical Sciences, Kerman, Iran

3 Medical Education Department, Tehran University of Medical Sciences, Tehran, Iran

4 Research Center for Hydatid Disease in Iran, Medical Education Department, Kerman University of Medical Sciences, Kerman, Iran

5 Kerman University of Medical Sciences, Kerman, Iran

6 School of Medicine, Physiology Research Center, Kerman University of Medical Sciences, Kerman, Iran

7 Shahid Beheshti University of Medical Sciences, Tehran, Iran

How to Cite: Dehghani M R , Salajegheh M, Fasihi Harandi M, Bahaadinbeigy K, Bahman Bijari B, et al. Design, Implementation, and Evaluation of a Medical Education Fellowship Program for the Faculty Members of Kerman University of Medical Sciences Based on the Kirkpatrick Model, Strides Dev Med Educ. Online ahead of Print ; 15(1):e64668. doi: 10.5812/sdme.64668.

ARTICLE INFORMATION

Strides in Development of Medical Education: 15 (1); e64668
Published Online: February 23, 2019
Article Type: Research Article
Received: December 2, 2017
Revised: September 29, 2018
Accepted: September 30, 2018
Crossmark

Crossmark

CHEKING

READ FULL TEXT
Abstract

Background: Attempts to increase the development of faculty members can improve their ability to assume different roles.

Objectives: The purpose of this study was to design, implementation, and evaluation a medical education fellowship program for the faculty members of Kerman University of Medical Sciences and to propose practical recommendations for the future design of development programs.

Methods: In this semi-experimental study, a total of 53 faculty members of Kerman University of Medical Sciences participated in a one-year development program, which was designed by the Education Development Center and included the main disciplines of medical education. The program was evaluated in several steps, using the Kirkpatrick model.

Results: In the first level of Kirkpatrick model, the majority of the participants were satisfied with the general quality of the fellowship program. Based on the findings, the program led to an increase in the knowledge of faculty members and promoted a more positive attitude towards education and these programs. The findings related to the second level of Kirkpatrick model showed a significant difference between the pretest and posttest results (P < 0.05). In addition, analysis of the effects of the program on the participants’ behaviors and practical learning indicated positive changes.

Conclusions: The medical education fellowship program led to positive changes in the participants’ attitudes towards education and faculty development programs and increased their knowledge about educational principles and strategies and achieving of training skills. It can be concluded that the medical education fellowship program could achieve many of its preset goals.

Keywords

Faculty Development Programs Medical Education Faculty Members

Copyright © 2019, Strides in Development of Medical Education. This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/) which permits copy and redistribute the material just in noncommercial usages, provided the original work is properly cited.

1. Background

The main purpose of medical education institutions is to provide human resources education and training and to facilitate health development in the country. Experienced and motivated faculty members are the most important elements in achieving this important goal (1). Efforts to promote the progress of these key figures in any educational institution can lead to an increase in the quality of learning, improve the learners’ capabilities, and increase the quality of health services (2). Therefore, the role of faculty members as leaders of students’ progress is very important (3).

Changes in healthcare organizations, increased complexity of healthcare services, and changing concepts of medical education, such as new teaching, assessment, and learning methods, have led to increasing demands for faculty members to fulfill different educational, research, leadership, and managerial roles (4, 5). Efforts and activities for the development of faculty members will strengthen their ability to assume different educational, managerial, and cultural roles and become role models in the field of health.

Many medical universities around the world have designed and implemented different programs to enhance the skills of faculty members (6, 7). The purpose of development programs for faculty members is to incorporate suitable activities in order to prepare and assist them in assuming different roles in education, research, leadership, and management. By using these programs, the individual’s productivity, efficiency, personal growth, and professional development are promoted for meeting organizational goals (4, 7-11).

In other words, faculty development programs are described as a process, which prepares the individual for different roles and helps them to be more productive and informed (12). Annually, medical universities spend significant amounts of money and time on the development of faculty members. Therefore, it is necessary to conduct an accurate and scientific evaluation to determine the strengths, weaknesses, and improvement strategies of these programs and to describe their overall effectiveness (13).

Various training courses, including the medical education fellowship program, have been used for the development of faculty members by the Education Development Center of Kerman University of Medical Sciences. This program was established in September 2014 with the aim of improving the faculty members’ knowledge about medical education and familiarizing them with new methods of curriculum and educational planning, teaching, and assessment. During two runs of the medical education fellowship program from 2014 to 2016, a total of 100 faculty members were participated.

2. Objectives

The purpose of the present study was to design, implementation, and evaluation a medical education fellowship program for the faculty members of Kerman University of Medical Sciences and to present practical recommendations for the design of future development programs.

3. Methods

In this semi-experimental interventional study, the educational development program for faculty members, developed by Education Development Center of Kerman University of Medical Sciences, was designed, implemented, and then evaluated. The development program continued for one year from September 2015 to September 2016. In this program, fifty-three faculty members of Kerman University of Medical Sciences were randomly selected among volunteers.

The educational content of the program included four modules as the main areas of education: Curriculum and educational planning, teaching methods, student assessment, and new technologies in e-learning. The duration of each module was two and a half months. The module of curriculum and educational planning was completed in four workshop sessions, each session was five hours (total, 20 hours), the module of teaching methods was completed in seven workshop sessions, each session was five hours (total, 35 hours). The module of student assessment was presented in four workshop sessions, each session was five hours (total, 20 hours), while the module of new technologies for e-learning was completed in three workshop sessions, each session was five hours (total, 15 hours).

The objectives of the module of curriculum and educational planning were as follows: Familiarity with curriculum and educational planning, acquisition of necessary skills for curriculum development, increasing the participants’ knowledge about the theoretical and practical frameworks of educational planning, and increasing their awareness about the current challenging issues of curriculum and educational planning. On the other hand, the purpose of the module of teaching methods was to help faculty members achieving skills and gain knowledge about modern teaching methods.

In the module of student assessment, considering the importance of learner assessment in the training cycle, develop a proper understanding of the importance and role of student assessment in learning, familiarize the participants with the concepts of educational assessment and different stages of exam planning, and promote their ability to design standard exams and evaluate the assessment system of medical students.

Finally, the goals of the module of new technologies for e-learning were familiarity with virtual education and the related programs.

The educational content of the sessions was selected by a panel of experts, based on reliable medical education resources and the literature on similar programs in other medical universities around the world. In our one-year fellowship program, a combination of student-centered and teacher-centered methods was applied. Therefore, a large part of the training was based on problem based learning, team work, projects, question and answer, discussions in small groups, e-learning, and virtual education.

The developed program was evaluated based on the Kirkpatrick model. In this model, there are four questions about the training program, which are addressed in each stage of evaluation (14):

Reaction: Did the learners show positive response to the program?

Learning: Did the program increase the knowledge of learners in a satisfactory way?

Behavior: Did the program cause positive changes in the behavior of learners?

Results: Did the program cause positive changes in organizational indicators?

The first level of Kirkpatrick model is reaction, which represents the learner’s interest in the course, desirability of the course, and level of satisfaction with the course. In terms of reaction to our fellowship program, the content, instructor, and facilities were evaluated using a 15-items questionnaire, which was presented to the participants at the end of the program. At the end of each questionnaire, the participants were asked to give their comments. The questionnaire was rated on a Likert scale, ranging from very weak (1 point) to very strong (5 points).

The second level of Kirkpatrick model is learning, which aims at defining and comparing the knowledge and skill of faculty members in the pretest (before the start of each module) and posttest (immediately after the end of each module). For this purpose, a four-option test, based on the program objectives and content, was designed, consisting of 20 questions. To validate this test, its content validity was confirmed by the panel of experts. In addition, its internal consistency was evaluated by calculating the Cronbach’s alpha coefficient (0.85).

In the third level of Kirkpatrick model, behavioral changes of the faculty members, participating in the medical education fellowship program, were examined. For this purpose, the scores of faculty members, given by their students before and after participation in the faculty development program, were compared. Finally, the fourth level of Kirkpatrick model is the results, which was not taken into consideration in our study due to the need for a time gap between program implementation and fourth-level evaluation.

The collected data are presented as mean and standard deviation. For data analysis, mean-comparison tests (t-test) were performed in SPSS version 23 (IBM Corporation, Armonk, NY, USA). P < 0.05 was considered statistically significant. During all stages of the study, ethical principles, including confidentiality of information, were considered, and informed consents were collected from the participants for completing the questionnaires.

4. Results

In this study, the medical education fellowship program was implemented at Kerman University of Medical Sciences, from September 2015 to September 2016, and a total of 53 faculty members were participated. Overall, 41% of the participants were clinical sciences, 25% were health and management, 12% were dentistry, 10% were basic sciences, 7% were pharmacy, and 5% were nursing and midwifery. Based on the findings, 64% of the participants had less than five years of work experience, while 36% had more than five years of experience. In terms of academic rank, 3% of the participants were instructors, 88% were assistant professors, and 9% were associate professors.

In the first level of Kirkpatrick model, most participants (87%) believed that the overall quality of the medical education fellowship program was satisfactory, while 12% described moderate satisfaction. Some participants found the program to be useful, interesting, and relevant to their goals; they also claimed that they became more aware of their strengths and weaknesses in education. In addition, reported higher self-confidence, motivation, and interest in teaching and stated that they would recommend the program to other colleagues.

Evaluation of the second level of Kirkpatrick model showed a significant increase in the participants’ knowledge from the pretest (mean score, 9.9 out of 20) to posttest (mean score, 13.9 out of 20), based on the results of t-test; this finding confirms the positive effects of the program on medical education.

In the third level of Kirkpatrick model, for assessing behavioral changes and application of acquired knowledge, the assessment scores of faculty members, given by 180 students, were compared before and after participation in the faculty development program. In the first semester, the mean assessment score of faculty members was 4.38 prior to participation in the program, while it increased to 4.5 in the semester incorporating the program. Moreover, the mean assessment score of faculty members was 4.8 in the first semester after the end of the program, which is indicative of positive changes.

5. Discussion

The purpose of this study was to design, implementation, and evaluation a medical education fellowship program for the faculty members of Kerman University of Medical Sciences and to present practical recommendations for the future design of faculty development programs. The results showed that this program led to positive changes in the participants’ attitude towards education, promoted the application of development programs for faculty members, and increased their knowledge about educational principles and strategies and acquisition of educational skills. Based on the results, the medical education fellowship program could achieve many of its preset goals.

Steinert et al., in a systematic review of articles published during 2002 - 2012 on the development of faculty members, concluded that the success of development programs is dependent on the accurate and scientific design of programs based on the needs of faculty members, with attention to the principles of adult education, interactive and collaborative methods of education, opportunities for gaining experience, and incorporation of long-term programs (5).

As mentioned earlier, design of development programs for faculty members should be targeted and in line with learning principles; additionally, participation of faculty members in the program should be facilitated (15). Therefore, in the design of our medical education fellowship program, we tried to develop the content and curriculum, based on the educational needs of faculty members and the main disciplines of medical education.

Generally, in the implementation of development programs for faculty members, it is necessary to use a variety of teaching methods in a flexible manner, depending on the conditions of the learner (16). In this study, for implementing the program, we attempted to use interventions according to the principles of adult education and learning, empirical learning, inclusive education, and feedback. In this regard, a study by Elliot et al., which aimed at examining the effectiveness of an 18-month faculty development program in medical education for faculty members, showed that the program could increase job satisfaction, motivation, self-confidence, and knowledge of faculty members about teaching and educational methods. Also, they concluded that faculty development programs need to be long-term in order to be successful (17).

Moreover, the results of a study by Knight et al. on the effects of a long-term development program for faculty members at Johns Hopkins University showed an improvement in the feedback skills of the intervention group and indicated their greater interest in student-centered teaching methods, compared to the control group (18). Similarly, in the present study, implementation of our one-year medical education program resulted in more stable results.

A review of the literature, including a review study by McLeod et al. (10), a systematic review by Steinert et al. (4), a research study by Sorinola and Thistlethwaite (8), a systematic review by Leslie et al. (1), and a review study by Steinert et al. (5), indicates significant positive changes in facultydevelopment programs over the past years. However, despite the progress in the number and quality of these programs, limited efforts have been made to evaluate their effects. Accordingly, in the present study, we aimed to evaluate the effects of a medical education program, using the first three levels of Kirkpatrick model.

The results of the present study are in line with the findings of some previous research (19-21), in which development programs for faculty members led to their progress. Nevertheless, there are limited studies evaluated the effectiveness of these programs in Iran, despite the establishment of numerous faculty development programs in our country. Similarly, no research has been carried out to investigate the effects of these programs in Kerman University of Medical Sciences. Therefore, in this study, the effects of a similar program were evaluated for the first time in this university.

Previous studies have reported consistent results to the present study. These studies indicated the participants’ satisfaction with faculty development programs, positive changes in their knowledge and attitude, and self-reported behavioral changes after participation in the program. In this regard, Kollisch et al. evaluated the effects of a development program about teaching methods for faculty members and reviewed the viewpoints of the participants, their assistants, and program organizers before and after the program. The results indicated that the faculty members had become more considerate of the educational needs of their assistants. They also showed increased flexibility, feedback, interest, and motivation for teaching after participation in the program (22). Moreover, Gelula and Yudkowsky concluded that participation in development programs would increase the knowledge of faculty members about teaching methods and strategies and improve their understanding of feedback compared to the pretest (23).

Studies by Berbano et al. (24), Hewson et al. (15), and Tax et al. (25) have suggested improvements in the faculty members’ ability to establish an appropriate learning environment after participation in the faculty development programs. In addition, they could ask more suitable questions, communicate effectively with the students, patients, and their families, and manage their time more efficiently. In addition, the teaching skills of the participants improved after the program, and positive changes were made in their attitudes towards faculty development and educational programs. Also, Behar-Horenstein et al. concluded that participation in such programs would result in the practical use of acquired skills, such as asking proper questions, student engagement in the classroom, and use of relevant examples for improving the understanding of materials (26).

Based on the results of this study, in the design of faculty development programs, it is suggested to implement long-term development courses based on relevant theoretical frameworks, on-the-job training principles, peer learning, and organizational support for these programs. In the evaluation of faculty development programs, it is also suggested to incorporate combined evaluation methods to review the changes after programs at organizational levels and to assess the extent to which the presented materials are practically used in the workplace.

There are several limitations in this study. First, the effects of the program were only examined in a small group of participants at a single university; this in fact limits the generalizability of our findings. Second, no control group was included in this study to accurately investigate the effects of the program; therefore, it was difficult to determine the exact effects and relate them to the medical education program. Finally, to investigate the results and outcomes of the program, three levels of Kirkpatrick model were considered in our study, while the fourth level was not taken into consideration due to the need for a time gap between the implementation of the program and the fourth level.

In future research, it is suggested to recruit a larger sample size from different universities. In addition, for a exact evaluation of the effects of faculty development programs, it is recommended to include a control group and perform comparative analyses.

5.1. Conclusions

Annually, many faculty development programs are established by medical universities for the development of faculty members; however, implementation of these programs is costly and time-consuming. Generally, the most positive outcomes of these programs include their efficacy in the education and training of learners and the consequent increase in the quality of provided services.

Acknowledgements

Footnotes

References

  • 1.

    Leslie K, Baker L, Egan-Lee E, Esdaile M, Reeves S. Advancing faculty development in medical education: A systematic review. Acad Med. 2013;88(7):1038-45. doi: 10.1097/ACM.0b013e318294fd29. [PubMed: 23702523].

  • 2.

    Steinert Y. Faculty development in the new millennium: Key challenges and future directions. Med Teach. 2009;22(1):44-50. doi: 10.1080/01421590078814.

  • 3.

    Wilkerson L, Irby DM. Strategies for improving teaching practices: A comprehensive approach to faculty development. Acad Med. 1998;73(4):387-96. [PubMed: 9580715].

  • 4.

    Steinert Y, Mann K, Centeno A, Dolmans D, Spencer J, Gelula M, et al. A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME guide No. 8. Med Teach. 2006;28(6):497-526. doi: 10.1080/01421590600902976. [PubMed: 17074699].

  • 5.

    Steinert Y, Mann K, Anderson B, Barnett BM, Centeno A, Naismith L, et al. A systematic review of faculty development initiatives designed to enhance teaching effectiveness: A 10-year update: BEME guide No. 40. Med Teach. 2016;38(8):769-86. doi: 10.1080/0142159X.2016.1181851. [PubMed: 27420193].

  • 6.

    O'Sullivan PS, Irby DM. Reframing research on faculty development. Acad Med. 2011;86(4):421-8. doi: 10.1097/ACM.0b013e31820dc058. [PubMed: 21346505].

  • 7.

    Steinert Y. Staff development for clinical teachers. Clin Teach. 2005;2(2):104-10. doi: 10.1111/j.1743-498X.2005.00062.x.

  • 8.

    Sorinola OO, Thistlethwaite J. A systematic review of faculty development activities in family medicine. Med Teach. 2013;35(7):e1309-18. doi: 10.3109/0142159X.2013.770132. [PubMed: 23464818].

  • 9.

    Steinert Y, Cruess S, Cruess R, Snell L. Faculty development for teaching and evaluating professionalism: From programme design to curriculum change. Med Educ. 2005;39(2):127-36. doi: 10.1111/j.1365-2929.2004.02069.x. [PubMed: 15679679].

  • 10.

    McLeod PJ, Steinert Y, Nasmith L, Conochie L. Faculty development in Canadian medical schools: A 10-year update. CMAJ. 1997;156(10):1419-23. [PubMed: 9164401]. [PubMed Central: PMC1227411].

  • 11.

    Steinert Y, Cruess RL, Cruess SR, Boudreau JD, Fuks A. Faculty development as an instrument of change: A case study on teaching professionalism. Acad Med. 2007;82(11):1057-64. doi: 10.1097/01.ACM.0000285346.87708.67. [PubMed: 17971692].

  • 12.

    Dent J, Harden R. A practical guide for medical teachers. 4th ed. Edinburgh: Churchill Livingstone; 2013.

  • 13.

    Gillespie M. Student-teacher connection: A place of possibility. J Adv Nurs. 2005;52(2):211-9. doi: 10.1111/j.1365-2648.2005.03581.x. [PubMed: 16164482].

  • 14.

    Rajeev P, Madan MS, Jayarajan K. Revisiting Kirkpatrick's model–an evaluation of an academic training. Current Sci. 2009;96(2).

  • 15.

    Hewson MG, Copeland HL, Fishleder AJ. What's the use of faculty development? Program evaluation using retrospective self-assessments and independent performance ratings. Teach Learn Med. 2001;13(3):153-60. doi: 10.1207/S15328015TLM1303_4. [PubMed: 11475658].

  • 16.

    Ramalanjaona G. Faculty development: How to evaluate the effectiveness of a faculty development program in emergency medicine. Acad Emerg Med. 2003;10(8):891-2. [PubMed: 12896893].

  • 17.

    Elliot DL, Skeff KM, Stratos GA. How do you get to the improvement of teaching? A longitudinal faculty development program for medical. Teach Learn Med. 1999;11(1):52-7. doi: 10.1207/s15328015tlm1101_12.

  • 18.

    Knight AM, Cole KA, Kern DE, Barker LR, Kolodner K, Wright SM. Long-term follow-up of a longitudinal faculty development program in teaching skills. J Gen Intern Med. 2005;20(8):721-5. doi: 10.1111/j.1525-1497.2005.0145.x. [PubMed: 16050881]. [PubMed Central: PMC1490179].

  • 19.

    McLean M, Cilliers F, Van Wyk JM. Faculty development: Yesterday, today and tomorrow. Med Teach. 2008;30(6):555-84. doi: 10.1080/01421590802109834. [PubMed: 18677659].

  • 20.

    Nasmith L, Steinert Y. The evaluation of a workshop to promote interactive lecturing. Teach Learn Med. 2001;13(1):43-8. doi: 10.1207/S15328015TLM1301_8. [PubMed: 11273379].

  • 21.

    Rust G, Taylor V, Herbert-Carter J, Smith QT, Earles K, Kondwani K. The morehouse faculty development program: Evolving methods and 10-year outcomes. Fam Med. 2006;38(1):43-9. [PubMed: 16378258].

  • 22.

    Kollisch D, Linsey S, Weiss JE. Using residents' ratings of teaching to assess the effectiveness of faculty development. Acad Med. 2000;75(5):558-9. [PubMed: 10824844].

  • 23.

    Gelula MH, Yudkowsky R. Using standardised students in faculty development workshops to improve clinical teaching skills. Med Educ. 2003;37(7):621-9. [PubMed: 12834420].

  • 24.

    Berbano EP, Browning R, Pangaro L, Jackson JL. The impact of the stanford faculty development program on ambulatory teaching behavior. J Gen Intern Med. 2006;21(5):430-4. doi: 10.1111/j.1525-1497.2006.00422.x. [PubMed: 16704383]. [PubMed Central: PMC1484783].

  • 25.

    Tax CL, Doucette H, Neish NR, Maillet JP. A model for cultivating dental hygiene faculty development within a community of practice. J Dent Educ. 2012;76(3):311-21. [PubMed: 22383599].

  • 26.

    Behar-Horenstein LS, Childs GS, Graff RA. Observation and assessment of faculty development learning outcomes. J Dent Educ. 2010;74(11):1245-54. [PubMed: 21045231].

  • COMMENTS

    LEAVE A COMMENT HERE: