Effect of student engagement on multimedia-assisted instruction

This study applied multimedia in a general engineering and technology course in Taiwan and evaluated the effectiveness of multimediaassisted instruction and learning. The course presented trends in technological development and the achievements of Taiwanese industries and research institutes from a historical perspective, and overviewed the technology industries and industrial transformation development in Taiwan. The course units adopted multimedia to support class teaching and student learning, and a survey was conducted to collect students’ attitudes and perception toward multimedia-assisted instruction and learning in the course. Research data were collected from 45 male and 9 female students with varied academic and cultural backgrounds. Results showed that multimedia videos help raise students’ awareness of learning issues, improve their understanding of content, and increase the depth of their learning. Almost all students liked the approach of using multimedia to assist teaching and learning, preferring this approach over traditional lecture-based instruction. They also would recommend this course to their peers. This study also found that the degree of students’ engagement caused variance in the students’ perception of multimedia helpfulness in assisting their learning. Finally, this study further proposes suggestions in both design and research on applications of multimedia-enhanced learning in engineering and technology education.


Introduction
In an era of unprecedented technological advancement, engineering practice has been continuing to evolve, but engineering education has not changed appreciably (Lang, Cruse, McVey, & McMasters, 1999).The Accreditation Board for Engineering and Technology (ABET) Engineering Criteria 2000, which was fully implemented in 2001, emphasizes outcomes over process, and provides an opportunity for stakeholders to help universities define educational goals and objectives and design a curriculum to meet the desired outcomes of advances in technology (Lang et al., 1999).Engineering education, which consists of science and mathematics based-subjects, is content-centered and design-oriented (Bourne, Harris, & Mayadas, 2005), and therefore the content of engineering education should change along with the developments in technology and industry.A general engineering education defines technology and the role it plays in society, develops students' problem-solving skills, and introduces students to a sequential design process that instills in them the ability to question a process or procedure.The goals of a general engineering education are designed to enable all students, even those who may not possess engineering training backgrounds, to adapt to the changing technological society of today and tomorrow (Mahajan, McDonald, & Walworth, 1996).For a general engineering education course, it is not easy for one teacher to prepare various materials to present different perspectives of engineering research and development to the class; on the other hand, in a scientific academic field, it is also hard for an engineering student to master all skills or knowledge domains.Therefore, having a variety of resources for main instruction to complement class learning is of special importance in a general engineering education.
To advance industrial development, the government of Taiwan established the National Applied Research Laboratories (NARL) in 2003, with the goals of establishing R&D platforms, assisting academic research, promoting frontier science and technology, and fostering high-tech human resources in Taiwan.It further produced videos to introduce advanced science and technology in Taiwan, such as circuit-related industries, the display industry, the computer and information technology industry, nano and biomedical technology, agriculture and ecological research, and traditional industry transformation development in Taiwan.The NARL video platform has become an online educational resource database that engineering educators can adopt into their daily educational practice.To full utilize these resources in a general engineering and technology education course entitled "Scientific Taiwan", offered in a research university in Northern Taiwan, the researchers, who were also instructors, modified and integrated these open resources provided by NARL into the course website, and demonstrated them in class teaching.To verify the effectiveness of this implementation, the purposes of this study are as follows: (1) to integrate the multimedia video releases from NARL into the instructors' teaching in the general engineering and technology education classroom to provide the students with a basic understanding of technological innovation and development in Taiwan; (2) to investigate the effectiveness of applying multimedia materials in assisting instruction and learning in a general engineering education course; and (3) to examine whether students' learning engagement in watching multimedia videos after class has an impact on their attitudes toward multimedia-assisted learning.

Multimedia-assisted instruction and learning in engineering education
Multimedia teaching integrates verbal material, such as printed and spoken text, and visual material, such as pictures, graphs, photos, and dynamic graphics (Mayer, 2001).Specifically video as a multimedia application has been viewed as powerful narrative visualization to be of benefit in teaching and learning in various disciplines including engineering, management science, physical, medical and science education (Garrison, 2001;Marx & Frost, 1998;Mitchell, 2001;Palmer, 2007;Shephard, 2003).In educational settings, including engineering education, multimedia presentations could provide variety to lesson planning, and facilitate instruction with better visualization and representation (Shephard, 2003;Teo & Chai, 2009).However, it does not easily and directly support active learning and to identify how to use multimedia-assisted instruction (MAI) effectively is a focus of many experts, professionals, and lecturers (Laurillard, 2002;Sung & Mayer, 2012).
Previous engineering education researchers have investigated effective teaching styles to fit different learning styles, and the relationships of the two types of styles, in order to find a better way to employ multimedia learning in engineering education.Multimedia approaches have been proved to have the potential to create a high-quality learning environment.Along with the rapid advances in technology, it becomes easier for instructors to access and apply multimedia to assist their teaching of engineering topics, and cases are well documented such as nanotechnology (Wishart & Blease, 1999), biotechnology engineering (Mutharasan, Magee, Wheatley, & Lee, 1997), circuit and electronic engineering (Maby, Carlson, Connor, Jennings, & Schoch, 1997;Schodorf, Yoder, & McClellan, 1996;Giannotti & Galletti, 1996;Höhne & Henkel, 2004;Nortcliffe & Middleton, 2008;Hunt, Howard, Kirk, Ash, & Tyrrell, , 2001), computer and information science, mechanical engineering, civil engineering (Kartam & Al-Reshaid, 2002), manufacturing education (Raju, Sankarz, & Xue, 2004), mechatronics (Kikuchi & Kenjo, 1996), mathematical education (Bringslid, 1999), and environmental engineering (Badran, 1995;Boman & Olsson, 1996).The advent of multimedia with hypertext, interactive simulations, and animation systems has forged an effective information delivery environment with the power of creating interactive laboratories and classrooms (Bengu & Swart, 1996).
Researchers have applied multimedia in educational contexts and found that it enabled them to choose from a range of media elements to convey a particular message (Cairncross & Mannion, 1999).The various forms of multimedia, such as videotapes (Mutharasan et al., 1997), multimedia materials on CD-ROM (Boman & Olsson, 1996), digital audio streams (Nortcliffe & Middleton, 2008), multimedia instructional software (Kikuchi & Kenjo, 1996;Schodorf, Yoder, & McClellan, 1996), web-based multimedia materials (Kartam & Al-Reshaid, 2002;Raju, Sankarz, & Xue, 2004;Bringslid, 1999;Badran, 1995), multimedia courseware (Giannotti & Galletti, 1996), and even the programmable multimedia systems (Höhne & Henkel, 2004), can meet different needs in practice.Previous studies have suggested that multimedia-assisted instruction can help teachers construct a coherent mental representation from presented materials (Mayer, 2001;Mutharasan et al., 1997), illustrate spatial relationships (Sutcliffe & Faraday, 1994), and present information appropriately for different kinds of learners (Cairncross & Mannion, 1999).It can also increase the depth of students' learning of specific issues (Höhne & Henkel, 2004;Cairncross & Mannion, 1999;Yueh & Sheen, 2009), and engage students in a way that static material does not (Cairncross & Mannion, 1999).Accompanied with good course design, it can also allow the instructor to quickly assess the effectiveness of a particular presentation (Maby et al., 1997).Felder and Silverman (1988) categorized learners by the way they receive information into three dimensions: visual learners, verbal learners, and kinaesthetic learners.They also suggest that instructors use graphical materials to accompany verbal materials in their class presentations to help both visual and verbal learners to learn in engineering courses.Using multimedia presentations to assist students' learning in engineering can help learners to recognize and retain the presented material (Cairncross & Mannion, 1999).But it also needs to improve their development in problem-solving (Höhne & Henkel, 2004) and decision-making (Raju, Sankarz, & Xue, 2004) with consideration of different learning styles and capabilities (Maby et al., 1997).Researchers have found that higher motivation (Giannotti & Galletti, 1996;Höhne & Henkel, 2004;Kikuchi & Kenjo, 1996) toward learning could lead to greater ongoing engagement when multimedia materials are used to assist learning (Nortcliffe & Middleton, 2008).Moreover, the better accessibility of multimedia-assisted instruction could also maintain and provide learning resources (Badran, 1995) for students to study at any time, place, and pace suited to the learner (Schodorf, Yoder, & McClellan, 1996;Kartam & Al-Reshaid, 2002).

Students' learning engagement of instructional activity in engineering education
In spite of the great interest in the multimedia to get learners' attention in the first place, relatively little evidence-base in the engineering education literature reveals its effect to support student learning with a focus on process and outcome.Various indicators and instruments that explore the concept of learning were examined and used in order to assess student learning, among them engagement refers to a student's psychological investment in, and effort directed toward, learning (Newmann, 1992).Students who are engaged with instructional activity are more likely to learn, to find the experience rewarding, to graduate, and to pursue higher education (Goodlad, 1984).Among a variety of indicators to learning, students' engagement is critical to student motivation during the learning process, and the motivation will then affect students' perception toward the materials, technologies and the overall instruction (Sutcliffe & Faraday, 1994).While the motivating strategies involve different levels of engaging (Ryan & Deci, 2000;Moller, Ryan & Deci, 2006), engagement can be viewed as a promising indicator to represent student learning process and outcome.Previous studies in video-assisted instructions have documented the difficulties and struggles to evaluate specific use of video in real sense (Bracher, Collier, Ottewill, & Shephard, 2005;Shephard, 2003).Due to the complexity involved in real educational settings and the nature of learning, it is not easy to precisely define any learning outcome to be affected by single media (Sung & Mayer, 2012), let alone most videos were used in the context of blended learning (Garrison, 2001;Palmer, 2007;Shephard, 2003).Therefore to verify what would affect student learning, this study adopted several conditional reasons through the educational context of applying MAI in general engineering education and further explored the effects of student engagement in multimedia-assisted instruction and learning.

Context
This study was implemented in a general engineering and technology education course entitled "Scientific Taiwan" offered by a research university in Northern Taiwan (See Fig. 1).The curriculum is specially designed for the foreign undergraduate students of various majors to learn about the general technological and industrial background of Taiwan.It is an elective course comprises 18 weeks of lectures, by 2 major instructors and 5 invited speakers from various disciplines at 4 universities and 4 research institutes in total.The contents covered significant achievements in the integrated circuit industry, display industry, computer and information technology industry, nano and biomedical technology, agriculture and ecological research, and traditional industry transformation development in Taiwan.Except for the 3 weeks of site visit, each week in the class meeting one or two technological topics were taught in lectures accompanied by a 30-minute multimedia video that presented relevant content to improve understanding of the issues (See Fig. 2).All of the video clips were pre-made by the instructors with English subtitles and Chinese narrations.After each class, these videos were streamed and posted on the course website for students to review anytime (See Fig. 3).

Instrument
The researchers developed a questionnaire in order to investigate the effectiveness of applying multimedia in support of instruction and learning in this general engineering education course.The questionnaire comprised three sections: information on gender, nationality, and specialty; students' experience of watching multimedia videos in support of their learning; and students' attitude toward the multimedia-assisted learning application.For the attitude survey, a 6-point Likert-type scale from strongly disagree (1) to strongly agree (6) was adopted for evaluation (Grover & Vriens, 2006).Based on a previous literature review, this study divided students into three learning groups according to their reasons for watching multimedia video after class.These students reporting autonomous reasons were defined as positive learners; those who reported heteronymous reasons were defined as passive learners; and those who never accessed the multimedia videos after class were categorized into the group of non-learning learners.
To ensure the validity of measurement of engagement, subject matter experts of engineering education were invited to validate the instrument.Reliability of the instrument was also reasonably accepted (Cronbach's alpha = 0.954).

Data collection and analysis
The questionnaire was administered to all students in the final class meeting of the course.
Descriptive and variance analysis were then conducted to understand the phenomenon of multimedia-assisted instruction.

Table 1
Background data of the participants (N=54) *More than one response was acceptable

Students' experiences of watching multimedia videos after class
There were 17 students (31.5%) who reported that they never watched the multimedia materials after class.For the other 37 students, the reasons they watched the multimedia videos after class included having to write class reports (44.4%);being interested in the lecture topics (27.8%); wanting to learn more (20.4%);wanting to review the contents learned in class (9.3%); and trying to catch up in the class (3.7%).The researchers further sorted students by their learning engagement in terms of their multimedia video watching behavior, and categorized students into three groups: 13 students that actively wanted to learn more as positive learners; 24 students who just wanted to meet assigned requirements as passive learners; and those 17 students who never watched the multimedia videos after class as non-learning learners.The students were asked to identify the top three lecture topics that they had enjoyed the most after class.Results showed that the topics of videos students preferred most were wireless technology in Taiwan (44.4%), nano education and core facilities (38.9%), and aerospace technology and defense technology (35.2%) (

Students' attitudes toward multimedia-assisted instruction
While this study tried to investigate students' attitudes toward multimedia-assisted instruction, results showed that they perceived the multimedia video as able to improve their understanding of the content (M=5.19,SD=1.01) and the depth of their learning (M=5.13,SD=0.99) in this course.They also perceived the multimedia video as able to raise their awareness of learning issues in this area (M=5.00,S.D=1.18).Students also reported their preference for multimedia-assisted learning.They liked this approach of using multimedia video to assist instruction and learning (M=5.17,SD=1.077) and preferred this approach over traditional lecture-based instruction (M=5.19,SD=1.011).Moreover, they would recommend this course to other students in the future (M=4.93,SD=1.226).Results of students' attitudes toward multimedia-assisted instruction confirmed the effectiveness of using multimedia videos in support of instruction and learning in this general engineering education course.Also, students in the positive learner groups showed the highest levels of appreciation for multimedia-assisted instruction on all items (See Table3  Attitudes toward multimedia-assisted instruction among the three learning groups

Students' learning engagement on their attitudes towards multimediaassisted learning
Results of the study showed that positive learners (M=5.54) had more positive attitudes toward multimedia-assisted learning than passive learners (M=4.95) and those students who never watched the multimedia video after class (M=4.94).In order to examine whether students' learning engagement in watching videos after class would have an impact on their attitudes toward multimedia-assisted learning, this study further conducted one-way ANOVA analysis at the 0.05 level to test the effect.Results revealed that some of the items did show significant differences among the three learning groups (positive, passive, and non-learning), indicating that this approach can improve the depth of their learning in this course.
To further examine the differences among the three groups, Schéffé's method was conducted as the post hoc comparison method.Results of analysis showed that the positive learning students of the former group had a greater perception of multimedia as able to improve the depth of their learning in this course than students who never watched multimedia videos after class (Table 4).Also students' nationalities and their general attitudes toward the multimediaassisted learning were compared to learn if the narrative visualization help foreign student develop better comprehension.However, it was found that the domestic students generally made higher appraisal toward the video on their learning than foreign ones, albeit of no significant difference (

Conclusions
This study intended to explore the effectiveness of applying multimedia in a general engineering and technology education course in support of students' learning.Results showed that students thought the multimedia could help them to improve their learning in general and in specific learning issues, which also confirmed the previous literature (Cairncross & Mannion, 1999;Giannotti & Galletti, 1996;Höhne & Henkel, 2004;Kartam & Al-Reshaid, 2002;Mutharasan et al., 1997), which indicated that applying multimedia to assist learning could help students to learn the materials.The introductory course introduced the technology and science developments in Taiwan, such as the integrated circuit industry, display industry, computer and information technology industry, nano and biomedical technology, agriculture and ecological research, and traditional industry transformation development.The three topics that the students showed the highest preference for were "Wireless Technology in Taiwan", "Nano Education and Core Facilities", and "Aerospace technology and defense technology".According to Badran (1995), multimedia materials should raise learners' awareness in certain aspects.The preference for related issues might lead students to further study on those issues in their later academic study.As for the effectiveness of applying multimedia materials in support of their learning, the students all liked this MAI approach, and preferred this approach over traditional instruction.Students also stated that it would increase their willingness to recommend this course to peers.
With further analysis, this study also found that students' degree of learning engagement caused variance in their perception of the helpfulness of multimedia-assisted learning in the course.Students who would actively review multimedia videos after class showed higher appreciation for this multimedia-assisted instruction.It would confirm the previous literature that the more students are motivated to learn, the more likely they are to perceive the effectiveness of instructional technology (Hunt et al., 2001;Kikuchi & Kenjo, 1996;Maby et al., 1997;Nortcliffe & Middleton, 2008;Sutcliffe & Faraday, 1994).The results of this study also showed, based on different degrees of learning engagement, a significant difference among the three groups in how much multimedia video could help improve their learning.The researchers therefore propose that engineering educators should be aware of the effect of positive student engagement on attitude toward learning, and should design more complementary learning activities such as multimedia assisted instruction, online learning assignment as well as collaborative teamwork assignments, which would help engagement students more in deeper learning.
As a case study, albeit important results were found and confirmed by this study, it should be noticed that within limited time and ethical approval to collect student data after class, the qualitative inputs of students were not included in the current study.It might result in a narrowing of the generalizability of the study, however it did highlight the need of future work when undertaking education research.It is suggested that except for focusing on the specific concept of learning effectiveness such as engagement in this study, future studies could include both qualitative and quantitative evaluations to reveal better empirical evidence that show students' characteristics in learning with multimedia, such as learning styles, abilities, and language proficiency, especially for a hybrid class of international students.It is expected that the results of this study should contribute to both engineering and technology education practice and future research.

Fig. 1 .Fig. 2 .Fig. 3 .
Fig. 1.The university general engineering education course designed for undergraduate students with a variety of nationalities Table 2).

Table 2
Students' experience of watching multimedia videos after class (N=54) *More than one response was acceptable ).

Table 5 )
. It was possibly due to the fact that the video clips were subtitled in English but narrated in Chinese, which might violate foreign