The coronavirus disease 2019 pandemic dramatically changed modern life. Traditional didactic lecture methods suddenly became impossible,1 and there was a need to maintain social distancing. A shift to online didactic lectures was the most common solution. However, there is evidence that information acquisition becomes inefficient beyond the first 10 to 15 minutes of a lecture.2 It may be even more difficult to concentrate in online lectures that lack interaction between the speaker and audience. Notably, videos longer than 10 minutes are less likely to be viewed.3 4 Short online video lectures (ie, micromodules), with or without interactive elements, offer an attractive alternative. Such micromodules can be incorporated into the flipped classroom (FC) model, which is a pedagogical paradigm shift that rearranges how time is spent in and out of the classroom.5

The FC model is becoming increasingly popular in medical education. It is attractive to the current generation of students who are accustomed to utilising digital media; on average, 70% of students prefer this learning model.6 Students can learn pre-class materials at their own pace; they can also enjoy more in-class active learning and interaction. Moreover, they can negotiate the FC platform at their preferred time and in their preferred place. Instead of passively delivering information in class, educators can devote valuable contact time to interactions with students, exploration of their needs, and discussions of more nuanced and challenging topics.6 The acquisition of foundational information becomes an active self-directed process, outside of the classroom.

Considering the continuous growth of medical literature, today’s medical students must acquire an expanded field of knowledge before graduation. A modern urology clerkship should alleviate the intense time pressures placed on students by helping them to effectively and efficiently develop diagnostic and procedural core competencies. Where possible, students should be allowed to learn by active participation, rather than listening and reading, during the limited available contact time. The FC model holds great promise in achieving this goal.7

The success of the FC model requires an efficacious online platform that facilitates self-directed learning; stringent evaluation of the online platform is necessary. However, methodologically rigorous qualitative and quantitative studies and evidence-based recommendations are scarce.8 Most published works quote practical wisdom, anecdotes, and principles of educational theory as the basis for their recommendations.9

This pilot study was conducted to compare our institution’s online micromodule platform with traditional didactic lectures in facilitating the acquisition of foundational urological knowledge by medical students.

This prospective, single-centre, single-blind randomised controlled trial, performed at a tertiary academic hospital, investigated whether online micromodules are non-inferior to traditional didactic lectures as an instructional medium; this trial is a component of a larger movement towards the FC approach in clinical training.

The urology clerkship is a surgical subspecialty in our faculty curriculum. All medical students have 1 week of clinical attachment in their final year of medical clerkship training (Year 6). The standard curriculum consists of lectures, bedside tutorials, and clinical shadowing. Traditionally, lectures are delivered to the whole class at the beginning of the academic year. Students then shadow our team in small groups on the wards, in clinics, and in the operating theatre. Teaching is opportunistic, based on symptoms, signs, investigations, diseases, and procedures encountered in the clinical setting. Formal knowledge assessment is conducted during end-of-year examinations in the form of written examinations (multiple-choice questions [MCQs] and short-answer questions), objective structured clinical examinations (OSCEs), and clinical short case examinations.

In this study, we selectively assessed knowledge transfer with regard to two urology topics: approaches to lower urinary tract symptoms (LUTS) and haematuria. First, a didactic lecture on the management of LUTS and haematuria, along with other topics, was recorded during its delivery in our routine lecture series for final-year students. Subsequently, two micromodules were prepared concerning the management of LUTS and haematuria; the micromodule content was similar to the didactic lecture content. The study participants continued with their scheduled urology training in Year 6; therefore, the study intervention was regarded as supplemental curriculum. Because the participants’ overall learning opportunities were not affected, we decided to obtain only verbal consent for inclusion in the study.

Medical students in Years 4 to 6 with no exposure to the urology subspecialty rotation were voluntarily recruited for the study. Participants were randomly allocated to either traditional didactic lectures or online micromodules; rigorous proctored tests were administered in accordance with the schedule shown in the Figure. Permuted block randomisation was conducted using a computer program. Random allocation sequences were placed into identical sealed and numbered envelopes. Designated research staff members were responsible for allocating consecutively numbered envelopes to the participants.

Students randomised to the traditional didactic lecture were grouped into a class, which watched the pre-recorded 45-minute didactic lecture in the lecture theatre (as if the students were attending a standard lecture). Students randomised to the online micromodule group viewed the micromodules on separate computers at their own pace. The total runtime of these micromodules was 10 minutes each, and students were expected to explore the content in its entirety within 45 minutes. The breadth and depth of topics covered in both interventions were identical to each other and similar to past lectures; the only difference was the delivery medium. The students could not be blinded; however, all outcome assessors (including content creators) were blinded to intervention allocation because the didactic lecture was not delivered live or in person.

We used the Kirkpatrick’s four-level training evaluation model as the basis for evaluations of instructional effectiveness. In the context of online learning, Level 1 (reaction) refers to the student’s affective responses to training quality or relevance, usually measured by surveys; Level 2 (learning) refers to knowledge directly obtained from the online lecture, usually measured by knowledge tests such as MCQs and true-false questions; Level 3 (behaviour outcomes/transfer of learning) refers to improvements in the outcomes of tasks not directly taught in the instructional content, typically measured through practical or standardised examinations; and Level 4 (results) refers to the impact of training on organisational goals (ie, actual benefit to patients).

Prior to randomisation, a pre-intervention MCQ test was used to determine participant baseline knowledge. Immediately after randomisation and completion of training, participants repeated the MCQ test to determine the degree of knowledge acquisition (ie, Kirkpatrick Level 2). Their confidence in the subject matter was also measured using a 10-point scale (ie, Kirkpatrick Level 1).

After 3 weeks of teaching, each participant underwent individual assessments in outpatient clinics. The MCQ test was administered again to test knowledge retention. Then, an OSCE was administered to assess the participant’s approach to a real patient with either LUTS or haematuria. A nurse was present as a chaperone and third-party assessor, who gave a subjective assessment score, measured using a 10-point scale. The participant then presented the case to a urologist, who assessed the collected information using a structured marking scheme. Additionally, the urologist gave a subjective assessment score, similar to the nursing assessment. All student assessors were blinded to the allocated teaching approach. The scores from the nurses and urologists were used to assess student performance in the OSCE (ie, Kirkpatrick Level 3); they also were used to assess the overall effectiveness and safety of the micromodule teaching approach. Due to the study design, the impact of training on organisational goals (ie, Kirkpatrick Level 4) was not assessed.

Statistical analysis was performed using SPSS (Windows version 23.0; IBM Corp, Armonk [NY], United States). There was no crossover between treatment arms. Data were analysed using an intention-to-treat approach. Descriptive statistics (means, standard deviations, and ranges) were used for demographic data. Independent samples t tests or one-way multivariate analysis of variance were used for parametric continuous variables; the Mann–Whitney U test was used for non-parametric continuous variables; and the Chi squared test was used for categorical variables. P values <0.05 were considered statistically significant.

Between 4 December 2017 and 22 January 2018, 45 medical students voluntarily enrolled in this study at our hospital; 22 students were randomised to the didactic lecture group and 23 students were randomised to the online micromodule group. Most participants (77% and 74%, respectively) were in their final year of medical education. There were no significant differences in demographic composition between the two groups (Table 1). The difference in pre-intervention MCQ scores also was not statistically significant (P=0.471), indicating that the participants had similar baseline knowledge (Table 2).

In this study, the primary outcome was the difference in OSCE scores between the didactic lecture and online micromodule groups, as assessed by the urologists. This outcome corresponds to Level 3 of the Kirkpatrick model. Three-quarters of participants assessed real patients with LUTS; the remaining participants assessed patients with haematuria. There was no difference in OSCE score between the groups (13.09 ± 1.59 vs 12.98 ± 1.75, P=0.823) [Table 2].

The secondary outcome was the difference in knowledge acquisition and retention between interventions. Knowledge acquisition was defined as the difference between pre-intervention and post-intervention MCQ scores. Knowledge retention was defined as the difference between pre-intervention MCQ score and pre-OSCE MCQ score (taken 3 weeks after the intervention). Both of these outcomes correspond to Level 2 of the Kirkpatrick model. There were improvements in MCQ scores after teaching in both groups, although not statistically significant. However, there was no difference in the degree of improvement between the groups. Therefore, knowledge acquisition for the two groups were similar. For the assessment of knowledge retention, one-way multivariate analysis of variance with adjustment for pre-intervention MCQ scores revealed no statistically significant difference between post-intervention MCQ score and pre-OSCE MCQ score (Wilks’ Lambda=0.894, P=0.101, partial η²=0.106).

Finally, subjective assessment of confidence and competence was conducted; this assessment corresponds to Kirkpatrick Level 1. There was a significant improvement in post-intervention self-rated confidence, but there was no difference in the degree of improvement between the groups (Table 2). In terms of clinical performance (Kirkpatrick Level 3), there were no differences between the groups in terms of subjective assessment score by the urologists (7.89 ± 0.91 vs 7.70 ± 0.91, P=0.487) or nurses (8.05 ± 0.72 vs 8.04 ± 0.71, P=0.993).

Our results show that both didactic lectures and online micromodules can help medical students achieve comparable outcomes in terms of acquiring, retaining, and applying foundational urological knowledge. Thus, online learning platforms may be viable substitutes for didactic lectures in the broader context of a move towards the FC approach.

In a systematic review of literature concerning the use of online lectures in undergraduate medical education,6 45 studies were identified; only 21 (47%) of those studies had clearly established control and intervention groups. Among the 21 studies, only six (29%) assessed learning using an OSCE or equivalent practical examinations; the remaining studies used MCQ tests. There was considerable heterogeneity in the manner by which online lectures were integrated into existing surgical curricula, which hindered meta-analysis. However, online lectures generally tended to be non-inferior to traditional lectures.

Online learning offers many benefits to educators and students. First, it ensures round-the-clock access to learning materials. Second, it allows students to revisit these materials throughout the curriculum. Third, online learning platforms can track and verify that students have accessed and completed specific materials. Fourth, electronic content can be updated in a convenient manner; distribution is instantaneous and universal. Fifth, students have autonomy over the sequence and pace of learning, as well as the allocation of time; these aspects allow them to tailor their learning experience to meet personal objectives. Sixth, although a higher initial investment may be required, online learning platforms can be reused, exchanged, and collaborated on; they offer new economies of scale.10 11 Finally, the coronavirus disease 2019 pandemic led to concerns about the spread of infection, such that online micromodules became an attractive option for medical student education that permitted social distancing. Notably, online micromodules represent easily accessible media that can be used for continuing medical education, and interactive teaching can be added to enhance learning experience.

An important limitation of online learning is that educators may utilise the scheduling freedom offered by online platforms to overburden students with learning materials; they may not consider the large amount of non-classroom time that may be allocated to other tasks. To avoid this phenomenon, we established ‘bite-sized’ modules (ie, micromodules) and ensured that all topics covered are highly relevant to future clinical practice. Such short modules also match the students’ attention spans.2 3 4 However, we acknowledge that educators may initially expend greater effort in the preparation of online modules.4

Although there were some improvements in MCQ scores after the lecture or micromodules, they were weaker than expected, potentially because the post-intervention MCQ test occurred immediately after the lecture and there was insufficient time for participants to process the lecture content. Another limitation of the study design was that there were no tutorials or in-class interactions after the lectures. Thus, the acquired knowledge may not have been consolidated, resulting in suboptimal knowledge retention. Nevertheless, this study was designed to demonstrate non-inferiority between pedagogical approaches. Educators should remember that online learning is one component of the overall FC model. An overhaul of the broader teaching mentality and existing curriculum is required to realise the paradigm shift offered by the FC model. Thus, simple conversion of existing lecture notes to an electronic format will not effectively facilitate learning. There is a need for full utilisation of software/technologies to prepare multimedia/truly interactive micromodules; this approach is more likely to enhance student learning experiences. It is also challenging to develop effective methods for assessment of student competencies. Educators should support and collaborate with clinicians in this regard, thereby complementing each other’s efforts.4 12 13 14 15

In addition to video lectures, online platforms can be used to deliver diverse educational content, including interactive multimedia learning modules, discussion forums, polling, and virtual patients. We deliberately excluded these materials for the duration of this study because they represent distinct instructional configurations in terms of content and interactivity. The combination of interactive elements and lecture into a single intervention group would have confounded and invalidated the results.6 8 Thus, the video lectures solely consisted of slide decks, narration, and video. More studies are needed to determine how to best incorporate these teaching approaches into the instructional design of future curricula.11

The present study focused on the transfer of clinical knowledge and management of common urological symptoms via micromodules. Future research should examine whether online lectures can also effectively transfer practical procedural skills. Because of time constraints and the curriculum system, exposure during the clerkship period is extremely limited. Therefore, the current instructional approach for physical examination and basic clinical procedures (eg, insertion of urethral and central venous catheters) is often informal, opportunistic, and unstructured. Further studies may clarify the role of online education in procedural training.

Online micromodules were non-inferior to a traditional didactic lecture in terms of knowledge transfer focused on urology topics. Further enhancement of the interactive elements of the instructional medium will improve learning experience. Micromodule utilisation can be optimised during the development of the FC model of teaching. In times such as the recent pandemic era, where social distancing must be maintained throughout the educational process, there is an urgent need for curriculum reform that maximises the use of technology to enhance medical student learning.

Concept or design: CF Ng.
Acquisition of data: CH Yee, PKF Chiu, JYC Teoh, FPT Lai.
Analysis or interpretation of data: K Lim.
Drafting of the manuscript: K Lim, CF Ng.
Critical revision of the manuscript for important intellectual content: All authors.

All authors had full access to the data, contributed to the study, approved the final version for publication, and take responsibility for its accuracy and integrity.

As editors of the journal, CF Ng and JYC Teoh were not involved in the peer review process. Other authors have disclosed no conflicts of interest.

This research received support from the Micro-Module Courseware Development Grant Scheme of The Chinese University of Hong Kong, Hong Kong (Ref No.: 3210817).

This research aimed to improve instruction through the use of educational tests administered to the participants. All participants provided informed consent without the collection of personal or sensitive data.

1. Al Samaraee A. The impact of the COVID-19 pandemic on medical education. Br J Hosp Med (Lond) 2020;81:1-4. Crossref

2. Mehanna WN. e-Pedagogy: the pedagogies of e-learning. Res in Learn Technol 2004;12:279-93. Crossref

3. Stuart J, Rutherford RJ. Medical student concentration during lectures. Lancet 1978;2:514-6. Crossref

4. Koo CL, Demps EL, Farris C, Bowman JD, Panahi L, Boyle P. Impact of flipped classroom design on student performance and perceptions in a pharmacotherapy course. Am J Pharm Educ 2016;80:33. Crossref

5. Rotellar C, Cain J. Research, perspectives, and recommendations on implementing the flipped classroom. Am J Pharm Educ 2016;80:34. Crossref

6. Cook DA, McDonald FS. E-learning: is there anything special about the “E”? Perspect Biol Med 2008;51:5-21. Crossref

7. Johnson L, Becker S, Estrada V, Freeman A. NMC horizon report: 2014 higher education edition. Austin (Texas): The New Media Consortium. Available from: https://www.learntechlib.org/p/130341/. Accessed 10 Mar 2020.

8. Sinclair PM, Kable A, Levett-Jones T, Booth D. The effectiveness of Internet-based e-learning on clinician behaviour and patient outcomes: a systematic review. Int J Nurs Stud 2016;57:70-81. Crossref

9. Cook DA. The research we still are not doing: an agenda for the study of computer-based learning. Acad Med 2005;80:541-8. Crossref

10. Tang B, Coret A, Qureshi A, Barron H, Ayala AP, Law M. Online lectures in undergraduate medical education: scoping review. JMIR Med Educ 2018;4:e11. Crossref

11. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. JAMA 2008;300:1181-96. Crossref

12. AAMC Institute for Improving Medical Education. Effective use of educational technology in medical education. Colloquium on educational technology: recommendations and guidelines for medical educators. 2007. Available from: https://store.aamc.org/downloadable/download/sample/sample_id/111/. Accessed 10 Mar 2020.

13. Sezer B, Abay E. Looking at the impact of the flipped classroom model in medical education. Scand J Educ Res 2019;63:853-68. Crossref

14. Hurtubise L, Hall E, Sheridan L, Han H. The flipped classroom in medical education: engaging students to build competency. J Med Educ Curric Dev 2015;2:35-43. Crossref

15. Hew KF, Lo CK. Flipped classroom improves student learning in health professions education: a meta-analysis. BMC Med Educ 2018;18:38. Crossref