The impact of flipped learning in surgical education: A mixed-method study

This study aims to investigate the effectiveness of flipped learning in surgical education in trauma. The participants were 15 first-year general surgery residents in Shiraz medical school. After completing a pre-test, the participants received the learning content in interactive multimedia, podcasts, movies, and books. One month later, they had a face-to-face class based on case-based discussions. An early post-test was performed immediately after the class, and a late post-test was performed one month later. In these tests, the residents' problem-solving abilities were assessed using multiple-choice questions. A semi-structured phone interview was used to measure their attitude towards the learning program. The results showed a significant difference between pre-test and early post-test scores and between pre-test and late post-test scores. However, there was no significant difference between early and late post-test scores. The residents were satisfied with the program's overall usefulness for junior residents. The participants tended to use podcasts more than other materials. They mentioned that the videos and books helped more to foster theoretical knowledge. The case-based discussion makes them more confident in dealing with patients in daily encounters.


Introduction
The education of general surgery residents is complicated. There are many expertise areas as a general surgeon, such as trauma which is considered a primary content area of general surgery education by the American Board of Surgery and other related committees (Musonza, Todd, Scott, Davis, & Potts, 2019). This subject might be more complicated, considering the residents' suboptimal training because of duty hour restrictions (Kairys, DiMuzio, Crawford, Grabo, & Yeo, 2009).
Trauma accounts for 20 percent of the global burden of diseases (World Health Organization, 2008). According to statistics, the trauma incidence rate is higher than the cumulative numbers of HIV, tuberculosis, and malaria each year (Mathers, 2008). This fact requires the immediate attention of surgical curriculum planners, mainly in the developing world, where 5-6 percent of mortalities are rooted in trauma occasions (Girgis & Miller, 2018). In the Emergency Room (ER), the patients' critical condition necessitates an emergent patient approach, obtaining a proper history and physical examination, precisely ordering first-line lab data, and starting management plans. Clinical management of trauma victims is a matter of golden seconds/minutes (Paydar & Akbarialiabad, 2020). In simple words, trauma is a surgical field where seconds matter and not minutes (Aggarwal et al., 2010).
Evidence shows that following the restriction in duty hours of residents and progression of subspecialty care, trauma training as one of the central core components of the general surgery curriculum has been markedly affected (Kuhls, Risucci, Bowyer, & Luchette, 2013). This requires adopting novel educational methodologies to enhance learning and compensate for the deficiencies of academic programs.
Moreover, the traditional concept of surgical education where the training occurs merely in the operation room context is questionable, as the surgical education has moved forward to enhance efficacy and decrease the risks (Kuhls et al., 2013). Also, residency training differs from other types of training. The pedagogical approach is a traditional education in which students learn, and teachers teach (Chacko, 2018). On the other side, in an andragogical approach, the mentor/teacher merely facilitates learning. This would promote self-directed, problem-oriented, and learner-centered learning. With such an approach, the role of didactic lectures is blurred, and more focus is on other types of learning, such as problem-based learning (PBL) (Amin & Khoo, 2003).
As a self-directed approach, flipped learning is composed of e-learning and faceto-face interactions. In this method, educational materials are provided to students outside the classroom through different technologies and modalities such as podcasts, video clips, multimedia, books, etc. Besides, face-to-face interaction focuses on interactive learning activities such as group discussion and questions and answers (Hew & Lo, 2018).
In this respect, while flipped learning has provided many learning benefits and opportunities for learners, it has also encountered challenges. A major challenge is the effective design of educational content and strategies. Combining active learning methods such as case-based learning with appropriate educational content increases the effectiveness of education. Notably, case-based learning is a teaching method developed around a case (Ali et al., 2018). Accordingly, it is possible to make mistakes and investigate the consequences in a safe environment, ultimately reducing the risk of medical errors in real situations (Nielsen, 2016).
Even though there is increasing interest regarding this subject in medical education (Hew & Lo, 2018), little has been published concerning flipped learning in the medical curriculum, precisely focusing on the post-graduate curriculum (F. Chen, Lui, & Martinelli, 2017). In the current study, we aimed to investigate the applicability of flipped learning in trauma education among junior general surgery residents in Shiraz medical school, Iran. To the best of our knowledge, this is the first report concerning the application of flipped learning in the trauma rotation of general surgery residents. We sought to answers four research questions: (I) Can flipped learning foster knowledge acquisition? (II) Can flipped learning lead to the encouragement of knowledge retention in trauma rotation? (III) How much our residents used each material provided? (IV) Was the overall program satisfactory to participants?

Literature review
The integration of any new educational strategy necessitates proper recognition of the pros and cons of the program (Jayakumar, Brunckhorst, Dasgupta, Khan, & Ahmed, 2015;Kew & Tasir, 2021). One novel educational method that integrates the traditional "inclassroom" education with e-learning is blended learning (Grasl et al., 2012). This method is considered a bridge from the conventional, teacher-dominated, and face-to-face lectures/discussions to fully web-based education (Back et al., 2014;Negahban & Zarifsanaiey, 2020). Fig. 1 presents the pros and cons of integrating e-learning in educational practice.

Fig.1. A schematic list of the advantages and disadvantages of e-learning
Blended learning, especially flipped classrooms, arose from high schools and undergraduates of non-medical fields. However, this method progressed rapidly in many areas, including medicine (Alberts, 2012). In this method, teachers provide the curriculum content in a balanced, targeted way. Some materials are provided in the form of movies, while others are offered in podcasts or books. Additionally, the students attend the case-based, teacher-facilitated discussions after reviewing and processing resources (F. Chen, Lui, & Martinelli, 2017).
This method was first used by Jonathan Bergmann and Aaron Sams in 2007 (Bergmann & Sams, 2007) when they decided to provide audio narrated PowerPoints for absent students in a chemistry course (McDonald & Smith, 2013;Tan, Brainard, & Larkin, 2015). They defined this kind of learning as "what traditionally done in class now is done in the homes, and what is traditionally done as homework now is done in the class" (McDonald & Smith, 2013). In this approach, as all students are, at least, familiar with the concept of the topic, the class time would be more beneficial to both high-ranked and mediocre ones (Prober & Heath, 2012). Also, as it is learner-based and discussionbased, it would promote higher cognitive functions such as analysis and critical thinking (Anderson & Sosniak, 1994).
Flipped learning has emerged in residency programs to address the busy schedules of faculties and residents and increase effectiveness considering the restricted duty hours (Girgis & Miller, 2018). This method is considered highly effective for junior residents because they are less experienced, quickly feel overwhelmed, and have high working stress, resulting in early burnout (Ebrahimi & Kargar, 2018). In designing a flipped curriculum for general surgery residents, teachers and curriculum planners should develop the curriculum in a time-effective and flexible manner that does not require onsite attending of classes (Wittich et al., 2018). Since the COVID-19 pandemic has disrupted in-person classrooms, electronic learning (and blended learning) is currently highlighted more (Akbarialiabad, Fard, et al., 2020;Akbarialiabad, Kavousi, Ghahramani, Bastani, & Ghahramani, 2020;Dhawan, 2020;Moszkowicz, Duboc, Dubertret, Roux, & Bretagnol, 2020).
A large number of studies have been conducted to investigate the effectiveness of flipped learning and have found this approach greatly effective in enhancing knowledge, skill, motivation, and satisfaction among students (K. S. Chen et al., 2018;Riddell et al., 2017). Moreover, Graham et al. revealed that flipped learning is more effective than conventional learning in knowledge acquisition among internal medicine residents (Graham, Cohen, Reynolds, & Huang, 2019). In a study considering undergraduate medical students, flipped learning was associated with higher satisfaction and engagement with teaching but not with the improvement of final scores (Chowdhury et al., 2019). Regarding dental education, a systematic review showed that flipped learning is an effective method of education and time flexibility was a major asset in this method, making students follow the learning materials with their own pace (Gianoni-Capenakas, Lagravere, Pacheco-Pereira, & Yacyshyn, 2019). In the case of nursing education, various studies showed the effectiveness of flipped learning in academic progress (Cho & Kim, 2019;Gu & Sok, 2020;Im & Jang, 2019;Lee & Park, 2018), and a meta-analysis confirmed this effectiveness (Li et al., 2020).
In light of these pieces of evidence, one can conclude that there is an increasing interest in implementing flipped learning in medical education curricula, and its encouraging impacts on academic improvements of students are not negligible. Since there is a scarcity of evidence in post-graduate and residency programs, we decided to investigate the effect of flipped learning on the first-year general surgery residents serving in trauma rotation. The findings can help convince medical education officials to integrate such learning methods into the teaching curricula.

Methods
The present mixed-method study (quasi-experiment & content analysis) with a one-group pre-test post-test design was conducted to determine the effect of a flipped learning program based on clinical decision-making and problem-solving ability of general surgery residents of Shiraz medical school, Shiraz, Iran, concerning the primary trauma survey. The census method of sampling was used, and participants were compared with their pre-test, early post-test, and late post-test outcomes.

Participants
First-year general surgery residents with Doctor of Medicine (MD) degree who took the trauma surgery rotation were eligible. Students who had no desire to complete the questionnaires and have had previous flipped learning experiences were excluded.
Fifteen participants were eligible at the time of the study (April -May 2019), nine were male, and six were female. All of them were invited to participate in the study. None of them met the exclusion criteria.
The Ethics committee approved the study of Shiraz University of Medical Sciences (IR.SUMS.REC.1399.721). After explaining the study's aims and method, all residents agreed to participate in our study. In this step, informed written consent was obtained from the participants. We assured confidentiality of information.

Learning materials
The study was designed with a collaboration between "Shiraz Trauma Hospital" and "Educational Development Center" of Shiraz University of Medical Sciences. The educational materials were composed of videos, rapid review podcasts, and books. The educational content was selected from two specific references: Advanced Trauma Life Support ( The multimedia content (video and podcasts) was prepared based on our most extensive consultation of two separate expert panels composing four trauma surgeons and five medical educationists, mainly according to the content of the ATLS. Curriculum and educational materials were prepared in June and August 2019. The recorded materials were 17 hours of video and five rapid review podcasts. The video sessions, using real-life scenarios, were comprise of 13 titles (each approximately 60 minutes and a maximum of 1.5 hours). The titles of the visual media are available in Table 1. The podcasts were about 30 minutes and rapidly reviewed the "must-know" points or videos. There was a one-session class reviewing all the essentials of the course in a casebased manner. It lasted for 2.5 hours and was held interactively. Clinical attendees discussed the cases separately and highlighted the must-know points of the course.
The implementation of the flipped education program and evaluation phase lasted from September to November 2019. In early September 2019, a pre-test examination was held to assess the residents' baseline knowledge or problem-solving ability regarding the primary trauma survey. After that, all participants received educational movies, podcasts, and books on the primary trauma survey. Four weeks later, in early October, a case-based discussion was held for residents in a 2.5-hour class discussion. In the discussion, the surgical attendees reviewed the topics using case-based teaching and interactive lecture. An early post-test was performed after the case-based discussion to assess the program's early effectiveness in clinical problem-solving. One month later (early November), all the residents participated in a delayed post-test examination evaluating the impact of the whole program on knowledge retention of the contents. In order to assess the higher level of cognition, pre-test, early post-test, and the late post-test were composed of 20 comparable clinical scenarios to evaluate clinical decision making and problem-solving capabilities (Armstrong, 2016). In mid-November 2019, a semi-structured phone interview, using a five-point Likert scale, was conducted by an external expert. Each interview lasted around 20 minutes. The interviews were then transcribed and analyzed using qualitative content analysis.

Problem-solving test
In order to investigate the residents' problem-solving ability, multiple-choice questions (MCQs) composed of 20 clinical scenarios on the primary trauma survey were employed. Each correct answer equaled one positive score for the responder with a total score of 20. The face and content validities were confirmed by four content experts of the "Educational Developmental Center" of Shiraz University of Medical Sciences. The reliability of the tests was approved with a correlation coefficient of 0.8, 0.75, and 0.78 using test-retest reliability assessment.

A semi-structured interview
The content validity of the structured interview form was validated by four content experts of the "Educational Developmental Center" of Shiraz University of Medical Sciences. The forms had five main sections with related subsections. The residents were asked to rate the whole program, how much they used each material, the benefits of each material in knowledge acquisition/retention, and confidence promotion in decision making. The final item was an open question to evaluate the benefits/drawbacks of this model. Phone interviews were transcribed verbatim and studied by the first and corresponding authors and analyzed with qualitative content analysis. The appropriate text fragments were chosen. Following that, the fragments were labelled and sorted. At the final step, utilizing these fragments, central themes were extracted. In cases where these authors were not in complete agreement, the final decision was made by the second author. Finally, eight main themes were extracted, which are available in Table 3.

Data analysis
Data analysis was carried out by SPSS statistical software (version 18, Chicago, IL, USA). The Mann-Whitney test, Spearman's rho, and paired T-test were used where applicable. P value < 0.05 was considered as the significance level.

Results
In this study, all the 15 (100%) junior general surgery residents completed the research process. In terms of gender and age, 60% were male, and 40% were female, with a mean age of 29 ± 3 years.
The first objective of the present study was to investigate the attitude of junior general surgery residents toward flipped learning. Another objective of the research was to determine the effect of flipped learning on gaining and retaining problem-solving ability in primary trauma surveys. In order to investigate the problem-solving ability, 20 multiple-choice clinical scenarios were used to assess the competency of residents' problem-solving in the pre-test, the early, and the late post-tests. Table 2, the results showed that in the quantitative phase, there was a significant difference between pre-test (M = 10.733, SD = 2.25) and the early post-test (M = 12.8, SD = 1.82) (p = 0.004), and between pre-test (M = 10.733, SD = 2.25) and the late post-test (M = 13.267, SD = 1.53) (p = 0.002). Interestingly, there was no significant difference between early post-tests and late post-tests (p = 0.404). Paired T-test Not significant (p = 0.404) Note. M = mean, SD = standard deviation Also, we correlated the test scores with the residents' gender using the Mann-Whitney test. There was no correlation between the test scores and gender (Table 3). The results of the test scores are available in the above tables. In each test, the mean and standard deviation of scores is provided. Performing the Mann-Whitney test, we found no correlation between the results and genders.

As shown in
In the qualitative phase, an external expert was called and interviewed the residents using a semi-structured form. The questions were mainly focused on the quantitative use of the materials, the overall gain of each material, promotion of theoretical knowledge, and bedside decision making. Moreover, they were asked to rate the program with a score of 1 to 5. The data are summarized in Table 4.
It is shown in 4 that the residents were satisfied with the program's overall usefulness (overall score 4/5). The average rate of video usage was 1.93/5. The video users believed that they promoted theoretical knowledge (3.33/5) and enhanced bedside decision-making (3.67/5). Regarding the podcasts, the mean employment of the podcasts among the residents was 5/5. The perceived gain from the podcasts regarding the knowledge and confidence in the management of the patients in real life was shown to be 4.33/5 and 3.6/5, respectively. In terms of the paper materials (books), the quantitative use of the books was 2.2/5. Besides, we found that the impact of the books in promoting theoretical knowledge was 4.13/5, while the impact of the books on the clinical judgment was 3.38/5. Concerning the case-based discussion session, all the residents attended that particular session (5/5). They believed that the efficacy of that case-based discussion on their knowledge and bedside judgment was 3.93 and 4.4 out of 5, respectively. Less use of the videos may be attributed to their long duration, which needs to be considered in further application of this method.

Table 4
Results from the semi-structured interview The overall score of the whole program 4 Efficacy of the case-based discussion in increasing the confidence in bedside decision making 4.4 Efficacy of the case-based discussion in promoting theoretical knowledge 3.93 Efficacy of the books in promoting increasing the confidence in bedside decision making 3.38 Efficacy of the books in promoting theoretical knowledge (out of 5) 4.13 Quantitative use of the books 2.2 Efficacy of the podcasts in increasing the confidence in bedside decision making 3.6 Efficacy of the podcasts in promoting theoretical knowledge 4.33 Quantitative use of the podcasts 5 Efficacy of the videos in increasing the confidence in bedside decision making 3.67 Efficacy of the videos in promoting theoretical knowledge 3.33 Quantitative use of the videos 1.93 Attendance in the case-based discussion All attended As summarized in Table 5, the final question in our phone interview was an open question. We asked the residents to mention any other required points to improve the program. After transcribing all the 15 interviews, the subjects were categorized into eight cardinal themes. Seven residents believed that the course was highly beneficial, and they suggested that this course be held every 2-3 months in a progressive mode. Six residents admired the case-based interactive nature of the discussion in helping them to overcome stress in real-life situations. However, one resident recommended that the session be over 2.5 hours to cover all the issues. Five residents suggested that the first month of the residency program be devoted to professionally prepare them to deal with patients, while two residents disagreed with this. The two residents held that education happens in real encounters of patients, and they preferred to have ongoing education while managing patients in clinical wards. In our qualitative study, five residents believed that podcasts are a great asset for learning, and they present a recapitulation of the essentials. They believed that they could listen to these audios while waiting for patients or going from one ward to another ward or in their leisure time. One resident suggested that the casebased discussion be held with the presence of senior residents, which helps the junior residents use their experiences. The company of the seniors could also facilitate the establishment of a peer mentorship between junior residents and the seniors (Table 5).

Table 5
The residents' attitude and suggestion toward the program

Statement
Resident number Regarding the timing, I suppose that the first month of our training program should be a pure observership with no clinical posting. In that time, we should learn/review all "must-know" points and then attend the wards. This will make us more confident when dealing with the patients 3, 4, 7, 11, 14 I suppose that we should not have a pure observership month filled with classes because we will notice the importance of the knowledge if we see their practicality in the field 12, 13 The case-based discussion was done in interactive mode and was helpful to decrease our stress of patient contact 2, 4,5,9,11,13,15 This method should be held every 2-3 months. The course should be progressive, starting with a primary trauma survey and going forward to more advanced care. 1, 2, 6, 8, 10, 11, 15 Podcasts were highly concise and too beneficial as we can review them when/where ever needed. 3, 10, 11, 12, 14 Videos were comprehensive and excellent for learning, but the duration of them should be more limited 4 The presence of senior residents as teacher assistants in CBD sessions may help us to have their experience as well and foster their mentoring role 8 The 2.5-hour class was not enough to review all essentials 3

Discussion
The study results showed that intervention improved the knowledge level at both followups. This finding is in line with other studies, which showed the effectiveness of flipped learning in terms of knowledge acquisition in medical education (Akbarialiabad, Fard, et al., 2020;Gianoni-Capenakas, Lagravere, Pacheco-Pereira, & Yacyshyn, 2019;Heitz, Prusakowski, Willis, & Franck, 2015;Presti, 2016;Rose et al., 2016;Tan et al., 2015;Tolks et al., 2016). The results reveal that flipped learning helps encourage deep and active learning (Danker, 2015;Lewis, Chen, & Relan, 2018) and school achievement (Liebert, Lin, Mazer, Bereknyei, & Lau, 2016). The flipped learning is increasingly gaining a reputation as a fascinating and successful instructional methodology. Evidence is still scarce in the utilization of flipped learning in medical education and especially surgical education and the field of trauma. This is the first paper which investigates the applicability of the flipped learning in the trauma for general surgery residents.
In our investigation, knowledge retention was confirmed by a lack of difference between the early post-test and the late post-test. We believe that this occurred probably because the provided materials were prepared according to daily encountered scenarios in our local trauma care center. As in our study, several surveys have proposed that this method promotes the test scores as compared with traditional methods (Wittich et al., 2018;Wolff, Wagner, Poznanski, Schiller, & Santen, 2015).
Similar to other studies, this study also showed a high level of satisfaction among participants. In a study by Tan et al.,96% of the respondents were satisfied with their flipped curriculum in emergency medicine and believed that is superior to traditional curricula. The participants believe that having the resources before attending the class encourages learning and is considered a motivator (Tan et al., 2015).
Moreover, in a cohort study, it was shown that besides experiencing a higher level of satisfaction, with flipped surgical curriculum, medical clerks were more interested in pursuing surgery as the prospective professional field. . Another survey conducted by Liebert et al. in a surgery clerkship showed that about 90% of students rated their flipped learning curriculum excellent or outstanding. Moreover, 84% of the clerks believed that other clerkships should adopt a flipped-based curriculum (Liebert, Mazer, Merrell, Lin, & Lau, 2016).
Advocating self-directed, active, and efficient learning are chief objectives for educational interventions for fostering adult learning (andragogy) (Kan et al., 2015;Kaufman, 2003;Tan et al., 2015). Other studies have shown the effects of flipped learning on metacognitive skills (Yilmaz & Baydas, 2017). In this method, the learners have the authority to manage resources, pace, place, and time, which makes them capable of self-regulation and performance (Lewis et al., 2018;Shyr & Chen, 2018;Tan et al., 2015). Our study found that the residents were delighted with their curriculum and uniquely interactive case-based discussion. They found it was constructive in relieving stress in real patient encounters. Another study confirms these findings as the participants believed that flipped learning and case-based discussion sessions could lead to a higher level of cognitive functions such as analysis and synthesis of knowledge (Moffett, 2015;Wood, 2003). Several papers concerning flipped learning support the effectiveness of case-based learning in these settings (Prober & Heath, 2012;Sharma, Lau, Doherty, & Harbutt, 2015;Srinivasan, Wilkes, Stevenson, Nguyen, & Slavin, 2007). Cases-based learning will foster higher-order educational objectives in bloom's taxonomy and cognitive ladder. We believe that this method is helpful as it encourages constructivism in medicine. Studies suggest that flipped learning helps learners build knowledge based on the pre-class materials, the case-based discussion, and their clinical experiences (McDonald & Smith, 2013;Prunuske, Batzli, Howell, & Miller, 2012).
One of our residents believed that the videos were lengthy but comprehensive. The videos lasted 17 hours on 13 subjects (with an average of 1 hour for each topic and a max of 1.5 hours). We believed, to be effective, all future video sessions should be limited to a maximum of 1 hour. This finding is consistent with other studies. The authors finally decided to trim the video materials to be less than 60 minutes, based on feedback they received at the end of the educational program (Kurup & Hersey, 2013;Sharma et al., 2015). Some experts believe that long videos can quickly fill the working memory, which has minimal capacity. However, shorter videos make the individual actively process ongoing information (Brame, 2015).
Also, we noticed that our participants tended to use podcasts more than other materials due possible to the remarkably shorter nature of podcasts, making them appropriate at any time and place. Our findings are similar to another study in which synopsized materials such as podcasts were more favorable to be used by the participants (Tan et al., 2015).
Our residents believed that videos and books help more to foster knowledge. The case-based discussion (CBD) makes them more confident in dealing with patients ( Table  2, 3). It may be implied that the high rate of the CBD is potentially due to the pre-class preparation of the participants, which encourages learning (Hew & Lo, 2018), being evidence for flipped learning. We noticed that none of the materials result in better results per se. Generally, they led to a significant difference in the efficacy of this method (Table  2). We believe the ideal framework should entail all of the materials and multiple strategies to cover numerous learning styles that meet the educational objectives of the program (Kannan & Kurup, 2012).
The high acceptability of flipped learning in this study revealed the efficacy and utility of this surgical field method, especially in trauma. In surgical residency programs, different schedules, limited educational time, and immediate need for the care of trauma victims threaten proper, uniform, and synchronized education of residents. Flipping the education will help the residents have an invaluable learning source, which is always available to them. This method also saves the time of instructors to repeat basic facts. It helps the surgical team focus on more advanced point-of-care teachings, which may be neglected due to time constraints (Tan et al., 2015).

Conclusion
We concluded that flipped learning is invaluable where the pace of attaining certain competencies matters like trauma. In our study, participants had a high level of satisfaction. The early and late post-tests were significantly higher in comparison to the pre-test, which was in favor of the efficacy of flipped learning in knowledge acquisition and retention.
Although our general surgery program is one of the most significant programs in Iran, this study was in a single school with a limited sample size. The small number of participants may mitigate biases, but further investigations are required. Also, the lack of a control group to compare the effectiveness was another limitation of this study.
Further cohort and randomized educational intervention trials with larger samples are required to investigate the applicability of flipped learning in post-graduates, mainly in general surgery and traumatology. Future studies might focus on mediator factors that could interfere with educational outcomes encompassing enthusiasm and motivation to learning, job satisfaction, job performance, job burnout, etc. The COVID-19 pandemic has disturbed regular modes of medical education. The promising effects of flipped learning in medical education highlight the possibility of its usage in medical curricula to encounter inevitable scars of the current pandemic on medical education.

Author Statement
The authors declare that there is no conflict of interest.