Investigating a blended learning context that incorporates two-stage quizzes and peer formative feedback in STEM education

: Researchers have expressed concern about the state of STEM education. To improve this situation, new pedagogies, such as blended learning, have been proposed and tested. The last decade has seen an increase in the use of blended learning to support learning; however, the effect of blended learning on learning remains unclear and often mixed. The two studies in this paper draw on data from pre-university science students in the following courses: (1) Electricity and Magnetism (E&M) and (2) Waves, Optics & Modern Physics (Waves). In study 1, the treatment group (blended learning coupled with two-stage quizzes & peer formative feedback) performed significantly higher than the control group (lecture format with online homework & instant feedback) in the standardized final exam. In contrast, in study 2, there was a non-significant main effect of groups, indicating that the treatment group (blended learning with online homework & instant feedback) and the control group (lecture format with online homework & instant feedback) performed similarly in the standardized final exam. The finding of study 1 suggests that the effect of an instructional pedagogical framework embedded in a blended learning context improves performance in STEM education. Whereas the finding of study 2 suggests that a blended learning context without incorporating any instructional framework or support for cognition other than the lecture is comparable to a traditional face-to-face course.


Introduction
The prominence of STEM (Science, Technology, Engineering, & Mathematics) education (see Akiha et al., 2018;Ardianti et al., 2020;Eagan et al., 2014;Hains-Wesson & Tytler, 2015;Koutsopoulos, 2019) has been recognized in the educational literature because of the essential role it plays in sculpting the knowledge economy, the workforce, and the technology industry.However, despite the importance of STEM training, the number of students pursuing STEM or completing STEM degrees continues to be a concern.Whereas the number of students interested in pursuing the STEM field is rising, there is a high attrition rate (Akiha et al., 2018;Eagan et al., 2014;Pryor & Eagan, 2013) once they enrol in colleges and universities.Overwhelmingly, 50% of STEM majors do not graduate or complete their bachelor's degrees within six years of entering college (Akiha et al., 2018;Eagan et al., 2014), and many of them drop out within the first two years of college (Watkins & Mazur, 2013).The attrition rate is even higher at the two-year college level, where two-thirds of STEM students do not graduate within four years (Van Noy & Zeidenberg, 2014).To address this problem, policymakers call for a 33% increase in STEM retention rates and more than one million STEM graduates by 2022 (Akiha et al., 2018;President's Council of Advisors on Science and Technology, 2012).
In the context of Quebec, the percentage of Collège d'enseignement général et professionnel (CEGEP -for a primer on CEGEPs, see Bazelais et al., 2016Bazelais et al., , 2018) ) students completing the two-year pre-university program within the two-year period is persistently low (31.70%) and only 65% complete their CEGEP education within four years (Service régional d'admission du Montréal métropolitain, 2016).Likewise, 30% of CEGEP students are permanent dropouts, with the dropout rate being 36% for males and 25% for females (Action Group on Student Retention and Success in Quebec, 2009).Furthermore, 54% of male students drop out before obtaining a CEGEP diploma, a percentage that is more than double the national average of 22% (Statistics Canada, 2008).Thus, it is imperative to examine why students drop out or shift away from STEM programs.
Given the ongoing concerns and challenges confronting the STEM fields -namely, poor instruction, lack of interest in STEM fields, declining rates of STEM degree production, lack of diversity, and high attrition rates (Akiha et al., 2018;Eagan et al., 2014;Hains-Wesson & Tyler, 2015;Watkins & Mazur, 2013) -researchers have increasingly sought for new pedagogical approaches and strategies to improve instruction, learning, and engagement in STEM education (Baldwin, 2009;National Research Council, 2011;Watkins & Mazur, 2013).As calls for STEM education improvements intensify, many have turned to new pedagogical approaches and technology to improve learning outcomes and achievement.Consequently, the use of new technologies in 21st-century teaching and learning is a topical theme in education (Lemay et al., 2019).Understanding how these technological tools can foster more profound learning experiences is critical for improving instruction and learning in STEM.In response, many new pedagogical innovations have been researched and offered.One unique offering is the blended learning (BL) approachthe combination of face-to-face (FTF) classroom instruction with online-mediated instruction (Graham, 2006(Graham, , 2009)), which has received growing attention from the education community (Bernard et al., 2014;Boelens et al., 2018;Im, 2021;Liu et al., 2016;Means et al., 2013;Spanjers et al., 2015).Indeed, educators have increasingly turned to the use of blended learning, and new developments in the implementation and use of blended learning continue to be documented (Halverson et al., 2014;Vo et al., 2017).
Despite the prevalence of blended learning and the increased interest in new pedagogical approaches to promote STEM education, there is limited research on the impact of blended learning in other educational settings, such as the CEGEP pre-university STEM program.Therefore, it is suggested that researchers should study pedagogical approaches before they can adequately account for the differences between blended learning and other contexts (Schmid et al., 2014) and, more importantly, how methods such as end-of-course assessment (Vo et al., 2017) impact students' learning practices and their academic performance.The present study explores whether a blended learning context coupled with an instructional pedagogical framework such as robust two-stage in-class quizzes and peer formative feedback leads to better performance on the standardized endof-semester final exam, particularly in the understudied pre-university CEGEP population.For added context, a two-stage assessment (e.g., quizzes) is defined as a pedagogical strategy that allows students to work on a quiz individually and where additional time is allocated to group discussions with peer formative feedback (see Bazelais & Doleck, 2018a, 2018b;Bazelais et al., 2019aBazelais et al., , 2019b)).

Literature review
Current research in science education argues that student-centred approaches such as interactive engagement (Caldwell, 2007;Hake, 1998) that actively construct new knowledge and meaning through mutually shared-understanding (Richardson, 2003;Hmelo-Silver & DeSimone, 2013) promote better learning experiences and learning outcomes in the college science classroom.Furthermore, research suggests that in-class interactive quizzes are an effective educational tool, as they tend to increase students' level of engagement, attention, interaction, and attendance (Dobbins & Denton, 2017;Kay & LeSage, 2009;Raes et al., 2020).Research also suggests that the inclusion of quizzes positively impacts the effectiveness and desirability of blended learning, more importantly, learning outcomes (Spanjers et al., 2015;Stockwell et al., 2015).Research suggests that a two-stage assessment can positively affect student learning outcomes by motivating the student's approach and attitude to study and learning (Bazelais et al., 2019a), and corrective feedback often provides valuable information on the correct solution (Butler et al., 2008;Butler & Roediger, 2008;Spanjers et al., 2015).
In addition, peer instruction and formative feedback promote the sort of support for cognition that encourages collaboration and more profound learning experiences (Elizabeth et al., 2012;Murphy et al., 2011;Watkins & Mazur, 2013).Research shows that low-stakes collaborative formative assessments (e.g., quizzes) coupled with peer interactive engagement improve academic performance when contrasted across diverse student populations in different class sections with various teachers (Haak et al., 2011;Roediger et al., 2011;Spanjers et al., 2015;Stockwell et al., 2015).More importantly, activities that cultivate collaboration and peer interaction are associated with better learning experiences and outcomes (Cooke et al., 2019;Menekse et al., 2013;Smith et al., 2009).One such activity is a two-stage collaborative assessment technique, whereby students are allowed to work on a quiz individually and then allowed to improve understanding through group discussions and peer formative feedback (Bazelais & Doleck, 2018a, 2018b;Bazelais et al., 2019aBazelais et al., , 2019b;;Cooke et al., 2019).Moreover, research suggests collaborative assessment techniques can improve retention and academic performance (Al-Sudani, 2020;Cooke et al., 2019;Gilley & Clarkston, 2014).
Comparative studies suggest that blended learning improved learning outcomes (Bernard et al., 2014;Drysdale et al., 2013;Means et al., 2013;Larson & Sung, 2009;Tamim et al., 2011).Blended learning can better address the needs of STEM students because it has the potential to create a more positive and interactive learning environment through increased engagement, collaboration, and time-on-task (Drysdale et al., 2013;Means et al., 2013;Martín-Martínez et al., 2020;Spanjers et al., 2015).Various research studies have concluded that collaborative or problem-based learnings are effective teaching and learning approaches for engaging students in blended learning (Keengwe & Kang, 2013;Stockwell et al., 2015;Yeh et al., 2011).Blended learning is promoted to meet the needs of diverse student populations (Spanjers et al., 2015;Yapici & Akbayin, 2012).Within this context, it is argued that it is more beneficial to make the education system less reliant on location and time (Raes et al., 2020;Spanjers et al., 2015;Yapici & Akbayin, 2012).

Purpose of the study
Although the literature suggests considerable affordances of blended learning, little is known about how such instructional strategies and practices affect learning outcomes in the context of understudied CEGEP pre-university science students.Specifically, there is scant research on pre-university CEGEP STEM students and on the effect of blended learning on academic performance in terms of end-of-semester assessment (Vo et al., 2017), such as a cumulative standardized final exam.
The overarching aim of this paper is to explore how a blended learning context coupled with two-stage quizzes and peer formative feedback affects students' performance in STEM education in an understudied pre-university population such as the CEGEP education system.The findings of this research study will broaden the scope of the effectiveness of blended learning in other educational contexts and provide encouragement for future implementations and research in post-secondary education.

Overview of the studies
This manuscript consists of two studies of pre-university CEGEP students that contrast learning outcomes from those in blended learning (treatment) versus traditional lecture format (control) classes, in the following science courses: (1) Electricity and Magnetism (E&M) and (2) Waves, Optics & Modern Physics (Waves).At the CEGEP where the studies were conducted, students enrolled in the 2-year pre-university science program are required to take three compulsory physics courses, namely, (1) Mechanics, (2) E&M, and (3) Waves.Mechanics is the first-semester physics course and introduces students to fundamental concepts and principles of Newtonian mechanics.E&M is the second physics course and introduces students to the fundamental concepts of electric charge, circuit networks, and electromagnetic fields.Waves is the third physics course and introduces students to the different types of oscillatory motion, waves, light, interference and diffraction, wave-particle duality, and quantum theory, including the special theory of relativity.These two courses have the same setup: 75 hours in total, which was divided into a 45-hour lectures and 30-hour laboratory periods per semester, including course evaluations.There are two 1.5-hour lectures (3 hour/week) and one 2-hour laboratory session per week.Upon completion of these compulsory courses, the student will have the required physics prerequisites for the pre-university CEGEP science program and university.Importantly, success in the latter physics courses (e.g., E&M and Waves) depends on an understanding of the underlying concepts introduced in the Mechanics course.
Study 1 contrasts two sections of E&M, where the blended learning context (treatment) incorporates asynchronous online video instructions coupled with frequent two-stage quizzes and peer formative feedback, while the control group uses the lecture format with online homework and instant feedback.A two-stage quiz is a pedagogical strategy that allows students to work on a quiz for 10-12 minutes individually, and an additional 10 minutes is allocated to group discussions with peer formative feedback.The two-stage quizzes were designed with key concepts or learning outcomes that students must know and learn.Accordingly, the two-stage quizzes comprised both conceptual multiple-choice questions and word problems, as was the case for the outside-of-class homework assignment for the control group.Overall, students spent on average 10-12 minutes on the two-stage quizzes individually, and then an additional 10 minutes were allocated to group discussions that allowed them to co-construct and share their understanding and give each other formative feedback.The two-stage quizzes were used to measure what students learned and understood from the course content and assess whether they could retain key underlying concepts throughout the semester.
Study 2 contrasts two sections of Waves, where the blended learning context (treatment) only incorporates asynchronous online video instructions with online homework and instant feedback, whereas the control group only uses the traditional lecture presentation with online homework and instant feedback.In addition, these studies sought to investigate the effectiveness of blended learning in the context of a standardized end-ofsemester assessment, such as a cumulative standardized final exam, including demographic characteristics such as gender in an understudied CEGEP population.

Study 1: The Electricity and Magnetism Physics course
A comparative study contrasts two sections (treatment and control group) of the Electricity and Magnetism (E&M) Physics course.In the blended learning format (treatment group), 40% of the FTF classroom lectures were replaced with asynchronous online video lectures.All the online videos, including the relevant lecture slides, were posted on the Omnivox portal with LEA (a course management system that allows for the distribution of documents and files, assignments, grade submissions, and discussion forums) with a notification and an allocated time frame in which the students must watch the videos, especially before the next class session.Rather than assigning weekly homework (as was the case with the control group), the students in the treatment group were quizzed ten times during the 15-week semester with no assigned homework outside of the classroom.Students were quizzed on average 3-4 times before each unit test, and the two-stage quizzes were designed not only to replace the out-of-classroom assignments but also to emphasize quality time-on-task and peer formative feedback.
In contrast, the traditional course with outside-of-class homework assignments served as the control group.The control group uses the lecture format to deliver the entire one hour and twenty minutes (1 hour 20 minutes) PowerPoint lecture with the aid of a SMARTBoard -an interactive whiteboard that includes a computer, a projector, and applicable software.The PowerPoint lectures were identical to both groups and were posted simultaneously to the Omnivox portal with LEA.In addition, as part of their course requirements, students were expected to do outside weekly reading from the required text and online homework assignments using LON-CAPA -an open-source e-learning platform that delivers personalized online assignments and instant feedback for each student.However, the control group had no weekly quizzes.Table 1 illustrates the two conditions in the present study.The outcome measures (quizzes, unit tests, homework, and standardized final exam (F.X.)) are also listed for each condition.

Control group
Lecture format combined with weekly outside-of-class homework assignments with no reduced FTF meetings Outside-of-class homework assignment, unit tests, F.X. Note.Each section was taught by a different instructor Different instructors taught both sections (treatment and control group) with identical content, and lecture slides, including three required unit tests (e.g., test 1 on week 5, test 2 on week 10, and test 3 on week 15) and a cumulative standardized final exam at the end of the semester.The three required unit tests and the standardized final exam were identical to both sections.In addition, both the treatment and the control group were assigned identical practice problem sets and a free lab session where students could work together, ask questions, and address areas of misconceptions with the co-presence of the instructor before each unit test.Each of the three-unit tests is weighted at ten percentage points for a total of 30% of their overall grade, while the standardized final exam is weighted at 40% overall.The remaining 30% comprises laboratory experiments (20%) and homework/quizzes (10%).The cumulative standardized final exam consists of 20 conceptual multiple-choice questions (20% weighted score) and 10-12 standard physics word problems (80% weighted score).
The course catalogue advertised the blended format, and students were informed that they were expected to participate in the FTF and online environments and have reliable internet connections.Students were also informed that they were expected to watch the online lectures by the due date or before the next classroom lecture session since they may contain critical concepts or assignments necessary and relevant for the next class session.In addition, participants of this study, both treatment and control groups, were informed of the confidential nature of the study and the data.They were assured that the study results would not be linked to any student's name or I.D..The data was not analyzed until the final grades were submitted.The research participants gave their consent to the researcher to assess and measure teaching and learning effectiveness using their aggregate quizzes, homework, unit tests, and final exam marks.

Study participants and procedure
The target population for study 1 is first-semester college physics students at an English CEGEP in Montreal, Quebec.The sample (N = 74, 51% males, 49% females) was drawn primarily from two sections of the E&M Physics course.The treatment group consisted of N = 36 students (44% males, 56% females), whereas the control group consisted of N = 38 students (58% males, 42% females).To rule out systematic bias, comparative statistics were used to analyze the sample and the two sections of E&M that were part of this study.No systematic differences between the two groups were found, as illustrated in Table 2.The High School Average (HSA) was essentially the same for the control group (N = 38, HSA = 83.53%,SD = 4.49) and the treatment group (N = 36, HSA = 85.39%,SD = 4.40).A one-way ANOVA shows that these two groups were not significantly different (F(1,70) = 3.33, p > .05) at the beginning of the semester.In addition, there was no significant difference between the genders (F(1,70) = 0.313, p > .05).

Analysis and results
An ANCOVA was conducted to examine the effects of the treatment (versus control group) and gender on final exam scores while controlling for the effect of prior knowledge by including High School Average (HSA) scores as a covariate.Using HSA as a covariate is congruent with several studies (e.g., Asarta & Schmidt, 2017;Gebara, 2010;Goode et al., 2018;Nielsen et al., 2018;Owston et al., 2013Owston et al., , 2020)).Preliminary tests were conducted to assess the assumptions of homogeneity of regression slopes and variance.The homogeneity of regression slopes was not violated (F(3, 67) = 2.69, p = .053),indicating no interactions between HSA and conditions.Additionally, Levene's test indicated that the homogeneity of variance was observed (F(3, 70) = .848,p = .472).After controlling for HSA, there was a significant difference in the standardized final exam results between the two groups (F(1, 69) = 4.298, p = .042,η 2 = .059).A post hoc analysis was performed with a Bonferroni adjustment indicating that the final exam result was significantly greater in the treatment group than in the control group (mean difference = 7.63, 95% CI [.288, 14.97], p = .0.042).While there were statistically significant differences between the two groups on the final exam result, there was no statistically significant effect of gender (F(1, 69) = .127,p = .723)or between group and gender (F(1, 69) = .062,p = .805),as illustrated in Table 3.A supplemental ANCOVA was conducted with HSA as the covariate and unit tests average as the dependent variable.After adjusting for HSA, there was no statistically significant difference in the unit tests averages between the two groups (F(1, 69) = .627,p = .431),as illustrated in Table 4.In addition, there was no statistically significant effect of gender (F(1, 69) = .011,p = .917)or between group and gender (F(1, 69) = .218,p = .642),on the unit tests average.

Study 2: The Waves, Optics & Modern Physics course
This comparative study was conducted to further test the effectiveness of blended learning that contrasted two sections of the Waves, Optics & Modern Physics (Waves) course.The treatment and the control group were taught by different instructors and used identical unit tests and weekly online homework assignments with instant feedback using LON-CAPA.The blended format (treatment group) course replaced 40% of the FTF course contents using asynchronous online video lectures without incorporating any instructional framework or support for cognition other than the lecture.The control group only used the lecture format as the primary mode of instruction.Furthermore, both sections had common learning outcomes and assessment methods.Table 5 illustrates the two conditions in the present study.The outcome measures (unit tests, online homework, and standardized final exam (F.X.)) are also listed for each condition.

Lecture format with online homework and instant feedback
Online homework, unit tests, F.X. Note.Each section was taught by a different instructor

Preliminary analyses
The target population for study 2 is fourth-semester college physics students, and the sample (N = 80, 49% males, 51% females) was drawn primarily from two sections of the Waves, Optics & Modern Physics (Waves) course.The treatment group consisted of N = 38 students (55% males, 45% females), whereas the control group consisted of N = 42 students (43% males, 57% females).To rule out systematic bias, comparative statistics were used to analyze the sample and the two sections of Waves that were part of this study.No systematic differences between the two groups were found, as illustrated in Table 6.The High School Average (HSA) was essentially the same for the control group (N = 42, HSA = 81.76%,SD = 2.68) and the treatment group (N = 38, HSA = 81.00%,SD = 3.38).A one-way ANOVA shows that these two groups were not significantly different (F(1,76) = 1.18, p > .05) at the beginning of the semester.In addition, there was no significant difference between the genders (F(1,76) = 0.00055, p > .05).

Analysis and results
An ANCOVA was conducted to examine the effect(s) of the treatment (versus control group) and gender on final exam scores while controlling for the effect of prior knowledge by including High School Average (HSA) scores as a covariate.Preliminary tests were conducted to assess the assumptions of homogeneity of regression slopes and variance.The homogeneity of regression slopes was not violated (F(4, 73) = .49,p = .742),indicating no interactions between HSA and conditions.Additionally, Levene's test indicated that the homogeneity of variance was observed (F(3, 76) = .50,p = .684).After controlling for HSA, there were no statistically significant differences in the standardized final exam results between the two groups (F(1, 75) = .021,p = .884).In addition.there was no significant interaction effect between group and gender (F(1, 75) = 1.14, p = .288),but there was a significant effect of gender (F(1, 75) = 5.66, p = .020,η 2 = .070),as illustrated in Table 7.A post hoc analysis was performed with a Bonferroni adjustment indicating that the final exam result was significantly greater for males than females (mean difference = 7.08, 95% CI [1.15, 13.01], p = .020).A supplemental ANCOVA was conducted with HSA as the covariate and unit tests average as the dependent variable, as illustrated in Table 8.After adjusting for HSA, there was a significant difference in the unit tests averages between the two groups (F(1, 75) = 14.80, p < .001,η 2 = .165),but there was no statistically significant effect of gender (F(1, 75) = 1.89, p = .173),or between group and gender (F(1, 75) = 2.251, p = .138),on the unit tests average.In addition, a post hoc analysis was performed with a Bonferroni adjustment indicating that the unit test average was significantly greater in the control group compared to the treatment group (mean difference = 10.06,95% CI [4.85,15.26],p < .001).

Conclusion
The overall findings of the two studies addressed the research question, that is, whether a blended college science course results in higher performance on the standardized end-ofsemester final exam in the context of a pre-university CEGEP science program.The empirical results of study 1 suggest that the effect of robust quizzes and peer formative feedback in a blended learning context improves STEM education, that is, superior learning outcomes and better performance in a cumulative standardized final exam at the end of the semester.There was a significant main effect of groups (F(1, 69) = 4.30, p = .042,η 2 = .060),indicating that the treatment group performed significantly higher (mean difference = 7.63, 95% CI [.288, 14.97], p = .042)than the control group in the standardized final exam.The findings suggest that a blended learning context leads to better long-term retention and more lasting learning outcomes.On the other hand, there were non-significant differences between the genders across all examined variables, indicating that both males and females performed equally in the standardized final exam.Furthermore, the findings of this empirical study suggest that the effect of frequent low-stake testing (e.g., quizzes) and peer formative feedback positively impact the effectiveness of blended learning.Similar findings were observed by Spanjers et al. (2015), who concluded that the inclusion of quizzes positively impacts the effectiveness and desirability of blended learning, more importantly, learning outcomes.It is suggestive that frequent testing can positively affect student learning outcomes by motivating the student's approach and attitude to study and learning (Bazelais et al., 2019a), and corrective feedback often provides useful information on the correct solution (Butler et al., 2008;Butler & Roediger, 2008;Spanjers et al., 2015).Within this context, it is suggested that quizzes and peer formative feedback increase both the effectiveness and magnetism of blended learning.
The positive effect of frequent testing (e.g., quizzes) (Adesope et al., 2017;Chen et al., 2018;Spanjers et al., 2015;Stockwell et al., 2015;Van Sickle, 2015) and peer formative feedback (Bazelais et al., 2019a(Bazelais et al., , 2019b;;Elizabeth et al., 2012;Murphy et al., 2011;Wentzel & Watkins, 2011) is well documented in the science literature.In the blended learning context, students could work individually (e.g., on in-class activities or quizzes) and then be afforded the opportunity for peer corrective feedback, co-construction of knowledge, and shared understanding.As the prevalence of blended learning receives widespread attention and becomes the "new normal" and calls for the need for greater student involvement and collaboration, self-regulation, and self-directed learning, it is, therefore, imperative for practitioners to provide students with the appropriate instructional content and strategies in the implementation of blended courses, for example, online activities, or video-embedded quizzes (Chen et al., 2018;Maciejewski, 2016;Murphy et al., 2016;Willis, 2014), in-class quizzes (Chen et al., 2018;Van Sickle, 2015), peer formative feedback (Bazelais et al., 2019a(Bazelais et al., , 2019b)).Furthermore, it is highly crucial that educators or practitioners of blended learning consider and rethink the instructional design according to the learner's need, employ a suitable methodology, use online activities and video-embedded quizzes or in-class quizzes to ensure that students remain engaged and committed to the online environment, and more importantly, perform the online assignments within the allocated period.Nonetheless, students' success in a blended learning format is perhaps contingent on whether they perform the online activities within the allocated period.
It is suggestive that an environment that provides students with the opportunity to solve problems with increased peer corrective feedback during class results in significantly higher learning outcomes and academic performance compared to simply having the same problems assigned to them as homework or answers described to them during the lecture.Furthermore, incorporating interactive engagement and peer formative feedback in a blended course can address many of the challenges students face in the traditional course.As a result, the blended course produces a more positive and active environment and enhances both the quality of instruction and learning outcomes while addressing the overarching concerns of poor teaching quality and low retention rates in STEM education.The results further demonstrate how adopting an effective blended learning context can enhance student performance and improve the quality of instruction in STEM-related programs.Furthermore, the findings suggest that blended learning methods can be adapted to foster quantifiable change and satisfaction in the science classroom, thereby, increasing student-student interaction, performance, and retention in the STEM field.
The findings of study 2 highlight that technology is a tool; when used effectively, it can improve the quality of instruction and learning outcomes (Alammary et al., 2014;Aycock et al., 2002).The non-significant finding of this comparative study (F(1, 75) = .021,p = .884,β = .052)reiterates the notion that technology is simply a tool, that is, technology is neutral and not necessarily decisive.In fact, technology is less effective unless it is integrated with a sound pedagogical framework or whether the teacher feels competence and confidence using that technology (Ertmer & Ottenbreit-Leftwich, 2010;McGee & Reis, 2012).The overall finding supports the idea that an effective blended learning context that incorporates an instructional framework or support for cognition is significantly more effective than those contexts without technology (e.g., FTF) or only using the technology as an add-on.Similar findings were observed in prior studies (Swoboda & Feiler, 2016;Tamim et al., 2011).The findings of study 2 suggest that simply putting videos or resources online does not necessarily lead to positive effects or outcomes.Furthermore, the findings further convey that thoughtful consideration should be taken when designing blended courses, especially in the context of instructional pedagogical design and implementations.It is suggestive that a blended learning context that simply replaced or redirected some elements of the FTF classroom environment with online videos without any support for instructional foundations or cognition resulted in comparable or equal learning outcomes and performance compared to a traditional course.

Limitations
The two studies are limited by their use of a non-randomized convenience sample and the fact that different instructors taught the treatment and the control groups.They are also limited because the instructors could not control textbook access, the amount of time students spent online viewing the videos, or whether the videos were used as review materials for the final exam.The findings of these two studies cannot be generalized given the small sample size, and the fact that the studies only contrasted two sections of the same courses during one semester.This is further supported by the non-significant gender differences and non-significant effects found in study 2, a blended learning context that simply directed videos (40% of the content) to the online environment without the assimilation of a pedagogical framework or support for cognition other than the lecture.A larger sample size with participants representing multiple sections of the same courses for more than one semester, including both the Fall and Winter semesters for both English and French CEGEPs in Quebec, could extend the findings.In addition, the cross-sectional nature of the studies limits the conclusions about continual knowledge acquisition or longterm retention.

Future directions
The present studies can be expanded by investigating a high-intensity blended learning context.Rather than modifying, replacing, or adding extra online activities or resources to the traditional course, alternatively, the entire course is built from the ground up, where a considerable proportion of the course content is directed to the online environment (Alammary et al., 2014).Furthermore, more research is required to investigate the proportion of time students spend in the online environment and whether the amount of time students spend online has any confounding effect on the effectiveness of blended learning contexts.It is suggested that a blended learning context that delivers over 50% of the course contents online has greater overall student satisfaction and performance (Owston & York, 2018).Research examining the predictors of blended learning effectiveness, for example, Kintu et al. (2017), find student characteristics/backgrounds and design features to be significant predictors of student learning outcomes.The present study can also be extended by examining different pedagogical approaches and design models in the context of blended learning and how these different models influence the pedagogical approaches to use and time spent online, impact learners' across cognitive, social, and affective dimensions in order to better understand the effectiveness and the transformative potentials of blended learning, and the association between blended learning and performance and satisfaction in terms of lasting, long-term learning gains, and retention.

Concluding remarks
By examining the overall findings of study 1 on students' learning outcomes, we found a strong positive effect of blended learning, however, this difference was non-significant between the genders.In contrast, the overall finding in study 2 reveals a non-significant effect for a blended learning context that does not employ a conceptual framework or support for cognition.As a result, this blended context is less effective; the learning outcome is comparable to or equal to the traditional course.Finally, we discussed the results and offered educational practice and research implications.The findings reported in the present study have broad implications for the blended learning literature, and that ongoing research leading to a better understanding of the relationship between blended learning and academic performance would benefit students and educators, practitioners, researchers, and university administrators.Consequently, this study provides researchers and practitioners of blended learning with a potential framework for applying and implementing blended learning models and designs in their teaching practice and research.

Table 1
Summary of methodology for study 1

Table 2
Summary of the sample, including overall high school average for each group

Table 4
ANCOVA analysis of test between subjects with HSA as the covariate and unit tests average as the dependent variable

Table 5
Summary of methodology for study 2

Table 6
Summary of the sample, including overall high school average for each group

Table 7
ANCOVA analysis of test between subjects with HSA as the covariate and final exam as the dependent variable

Table 8
ANCOVA analysis of test between subjects with HSA as the covariate and unit tests average as the dependent variable