Title Design and evaluation of a Facebook game for self-directed e-learning

Social networking sites (SNSs) such as Facebook have a potential to become a valuable learning environment. Facebook games with appropriate instructional design may provide players with better learning experiences and outcomes. Using an effective educational Facebook game, we aimed to explore the educational effects of Facebook games as self-directed e-learning environments. We tested our hypotheses on a sample of 73 undergraduates (42 females). The participants completed the Facebook game and self-administered questionnaires over a 3-week period. Path analysis demonstrated that Internet self-efficacy, usability, and fun positively affected perceived learning effectiveness and user satisfaction in a Facebook learning environment. We discussed the research and practical implications of these findings for the future development of self-directed e-learning on SNS.


Introduction
While SNSs became more and more popular, educators and researchers have explored the possibility of the educational usage of SNSs (Kabilan, Ahmad, & Abidin, 2010).Based on a critical review on Facebook as a learning environment, almost all studies identified in the review (22 out of 23 articles) evaluated Facebook groups or Facebook pages on group discussion, project collaboration, and peer assessment in education, whereas only one study investigated the educational usage of Facebook applications (Manca & Ranieri, 2013).Since a Facebook group can be set as private for privacy and security reasons, students can share learning resources within their study groups, and teachers can deliver course contents to students conveniently.However, the findings for Facebook group or page as a possible and effective learning environment in the review were mixed.On the other hand, the educational usage of Facebook application remains unexplored in literature.Since developing educational Facebook applications involves programming such as using the Facebook application programming interface, using an alreadyavailable Facebook group is much more convenient and popular than programming a Facebook application for most educators.
However, Facebook applications nowadays, especially social games, emerge and attract a large amount of people to play every day.For example, many adolescents spend much time growing crops in Happy Farm and feeding pets in Pet Society on Facebook.Therefore, Facebook games may provide institutions a great opportunity to engage students for not merely entertainment but also educational purposes for the mass (Yonker, Zan, Scirica, Jethwani, & Kinane, 2015).Educators and researchers have been searching for effective ways to integrate games into education, in which formal classroom-based learning and assessments are often considered by students to be a boring activity (Kiili, 2005).Research has shown that digital game-based learning (DGBL) can facilitate teaching and motivate students in learning processes (Girard, Ecalle, & Magnan, 2012;Papastergiou, 2009).However, few studies have fully utilized the Internet to provide larger learning flexibility.
The current study attempted to examine a novel approach of an educational Facebook game that is learner-centered and self-directed so students can play and learn to achieve as much as they can anytime and anywhere as long as Internet access is available.The Facebook game in the study is different from previous DGBL because the Facebook game also facilitated interactions among learners.For example, learners in the Facebook game can post their learning progress on Facebook timelines to share with their Facebook friends.Since social motivations lead to satisfaction with SNS use (Kim, Kim, & Nam, 2010), user interaction in the Facebook game, which is believed to be absent in previous DGBL, may enhance motivation of learning and hence facilitate self-directed learning.Based on an effective educational social game in enhancing mental health literacy (Li, Chau, Wong, Lai, & Yip, 2013), the study contributed to existing e-learning literature in two ways: (1) it explored the educational effects of using Facebook games as selfdirected e-learning environments, and (2) it examined the associations among Internet self-efficacy, usability, fun, user satisfaction, and learning effectiveness in the context of Facebook.

Digital game-based learning
Nowadays, young people are "digital natives" who prefer games over "serious" work (Prensky, 2003).Educators have added gaming elements to make learning more enjoyable (Garris, Ahlers, & Driskell, 2002).Squire (2003) suggested that games are powerful educational tools because they elicit strong emotional reactions within learners, such as fear, power, aggression, wonder, and joy.Educators have developed different frameworks and models on the use, selection, design, and development of educational games to ensure that learning outcomes will be consistent with the learning objectives of the games (Dondlinger, 2007).The effectiveness and perceptions of students of DGBL have been gradually revealed by substantial studies that make DGBL increasingly important in pedagogy (Bourgonjon, Valcke, Soetaert, & Schellens, 2010;Papastergiou, 2009).
There are different aspects to investigate the learning effectiveness of DGBL, such as the learners' self-efficacy (Gangadharbatla, 2008) and game quality (Hart, Ridley, Taher, Sas, & Dix, 2008).Different learning approaches and game designs may facilitate DGBL in different ways.Simulation, which is one of the most widely used approaches, provides a gamelike environment for learners to practice and acquire professional and experimental skills (Chau et al., 2013a;Chau et al., 2013b).Simulations have been shown to enhance learners' problem-solving skills and refine their higher-order thinking strategies (Douma, van Hillegersberg, & Schuur, 2012).Besides, narrative can support problem solving in adventure games as "players are placed in scenarios in which they must synthesize diverse information and analyze strategies" (Dickey, 2006).

Social games and self-directed e-learning
The Web 2.0 concept has led to the development and evolution of web-based communities (Chau & Xu, 2012) and applications such as SNSs, video-sharing sites, blogs, forums, Wikis, and highly interactive games.With the rapid development of SNSs such as Facebook, many people have created accounts and become active on these social and interactive platforms.For instance, Facebook users can interact with friends by commenting, liking, tagging, and sharing posts and photos on timelines.They can also join different groups, events, and pages.Different Facebook-related behaviors have been studied (Gosling, Augustine, Vazire, Holtzman, & Gaddis, 2011), and the possibility of integrating Facebook and education has also raised researchers' attention (Manca & Ranieri, 2013).Special and Li-Barber (2012) have found that relationship maintenance was the strongest motive for using Facebook.Social trust was also found to increase when the intensity of Facebook use increases (Valenzuela, Park, & Kee, 2009).Aside from that, students are generally open to use Facebook as a learning environment (Roblyer, McDaniel, Webb, Herman, & Witty, 2010).
Many studies have explored the educational usage of a Facebook group instead of a Facebook application (Manca & Ranieri, 2013).It has been evident that a Facebook group is conducive to group discussion, project collaboration, and peer assessment in learning processes.Education on Facebook inherently allows students great flexibility and interactivity while learning (Yonker et al., 2015).However, similar to other elearning approaches, education on Facebook requires learners to have a high level of selfregulation (Devolder, van Braak, & Tondeur, 2012).Therefore, a Facebook application may lead to an evolution of e-learning, with the addition of social gaming concepts to increase learning motivation, and become an increasingly relevant trend.A large amount of social games on Facebook, which are a kind of Facebook application, has emerged to make the platform more enjoyable and, most importantly, facilitate interactions between friends.Some of the social games available on Facebook are educational.For instance, Gameloft teaches American history, poweRBrands teaches business and marketing, and InGenius is a brain-training game teaching people how to think (Mack, 2010).However, few have evaluated the learning effectiveness and user satisfaction of educational Facebook games.

Internet self-efficacy
Internet self-efficacy reflects one's judgment of his/her ability to apply Internet skills, such as finding information or troubleshooting search problems on the Internet.Researchers have claimed that people with high Internet self-efficacy demonstrated better informational search skills and online learning achievement in many studies of webbased instruction (Eastin & LaRose, 2000;Joo, Bong, & Choi, 2000;Tsai & Tsai, 2003).Facebook usage may be related to Internet self-efficacy (Gangadharbatla, 2008).We expected that the same principles would apply to the educational use of Facebook, and thus, we hypothesized the following: H1a: Individuals who have a higher level of Internet self-efficacy will learn more effectively in the Facebook game.
H1b: Individuals who have a higher level of Internet self-efficacy will be more satisfied with the Facebook game.

Usability
We define usability as the ease of use and learnability of the Facebook game.Usability in Facebook plays a new role compared to its traditional function (Hart et al., 2008).Usability can affect players' learning effectiveness.For example, poor usability may break the harmony of an educational game and adversely affect user experience (Kiili, 2005).Therefore, users may need to apply extra effort to manipulate a learning tool and pay less attention to important learning materials.Besides, user satisfaction was jointly determined by perceived usability, perceived quality, perceived value, and usability disconfirmation (Chiu, Hsu, Sun, Lin, & Sun, 2005).Therefore, we hypothesized that: H2a: Individuals who consider the Facebook game to have better usability will learn more effectively.
H2b: Individuals who consider the Facebook game to have better usability will be more satisfied with the Facebook game.
Prior experience may facilitate one's control of an educational tool.Participants may consider the Facebook game easier to use if they have a higher level of Internet selfefficacy.Few studies have investigated the relationship between usability and Internet self-efficacy in the context of Facebook.Therefore, we hypothesized the following: H3: Individuals who have a higher level of Internet self-efficacy will consider the Facebook game to have better usability.

Fun
In the study, we defined fun as playfulness and emotional reactions of enjoyment and excitement (Hart et al., 2008).The theory of fun has recently emerged in game design theory and attracted much attention from both educators and researchers (Koster, 2010).Designing for fun can make interactive gaming elements change human behaviors and hence facilitate learning processes.Few studies have investigated the direct association among fun, learning effectiveness, and user satisfaction.Although significant association between fun and intention to use has been found in previous studies (Lee, Cheung, & Chen, 2005), intention to use may not reflect a comprehensive learning experience in the Facebook game as learners may continue to play a game without learning anything.Few studies have examined the association between fun, learners' satisfaction, and learning effectiveness in education on Facebook.Therefore, we hypothesized the following: H4a: Individuals who have more fun in the Facebook game will learn more effectively in the Facebook game.
H4b: Individuals who have more fun in the Facebook game will be more satisfied with the Facebook game.
A better e-learning environment facilitates participants' learning experiences.The association between enjoyment, playfulness, and ease of use has been explored in previous studies (Venkatesh, 2000).Participants may have more fun and enjoy the gaming element of the environment if they can easily and comfortably manipulate the Facebook game.Therefore, we hypothesized as follows: H5: Individuals who consider the Facebook game to have better usability will have a higher level of fun in the Facebook game.

Participants
An invitation e-mail was sent to all undergraduate and postgraduate students (N=22,260) at a university in Hong Kong, inviting them to participate in the current study.The sample included undergraduate students (N=73; 42 females; mean age=20.82;SD=1.81) who were existing Facebook users.They agreed to participate in the study and completed the Facebook game.An ethical clearance for research was obtained from the Human Research Ethics Committee for Non-Clinical Faculties at the authors' institution before data collection.
A household survey in Hong Kong revealed that nearly all youths (99%) claimed to be an Internet user (Hong Kong SAR Government, 2012), making the Internet a scalable content delivery medium that is particularly well suited for Hong Kong since it allows users to conveniently and privately access e-learning resources from anywhere with Internet access.Furthermore, 96% of the youth are engaged in online social networking.
Participants who completed the game and the questionnaire received cash compensation and were automatically entered into a "lucky draw," with supermarket cash coupons and electronic products as incentive boosters.

Game and instructional design
The Facebook game was a role-playing game based on a narrative adventure (Dickey, 2006;Prensky, 2003).The main character, Ching Ching, faces different tasks in the story, and the players solve the problems from a first-person perspective.To avoid overloading players with extensive learning materials, the whole game was divided into ten missions with a storyline (as shown in Fig. 2): (a) identifying stressors and how to handle stress, (b) understanding the relationship between stress and coping and the consequence(s) of depression, (c) understanding goal-directed thinking, (d) affirming existing strengths and acknowledging the concept of "self," (e) cognitive restructuring, (f) advanced cognitive restructuring, (g) understanding other people's feelings, (h) communication skills, (i) conflict resolution (based on a problem-solving approach), and (j) anger management (Wong et al., 2012).Players could determine the order of the missions.Some missions, however, were more challenging or required knowledge learned from other missions.Those missions were locked until prerequisite missions were completed.Different skills could be learned from different missions, with the skill level increasing with continuous practice (see Fig. 3).A certain amount of the character's energy was consumed after working on some tasks.When there was no energy left, the player could not advance the plot until the energy was recovered to prevent ineffective learning.Fig. 4 shows the view of Ching Ching's energy bar that the gamers saw.If Ching Ching used too much of her energy, this bar would drop to zero, and she would need sleep.The character's energy would resume either after a certain amount of sleeping time had passed or after she had consumed recovery items.
We attempted to balance extraneous contexts, such as background music or storylines, and learning efficiency.For example, the user interfaces reduce substantial extraneous processing but provide multiple functions for players to more conveniently control Ching Ching.Fig. 4 shows a list of buttons located at the right side of the screen that were used for the following actions:


The "Save" button allowed the player to save the game at any time.


The "Map" button allowed the player to go directly to four places to complete tasks.


The "Skills" button showed the player's skill level.


The "Items" button showed the player's acquired items.


The "Tips" button provided hints or clues to players so that they could easily complete tasks.


The "Home" button allowed the player to go back to Ching Ching's home.


The "Message" button indicated whether there were any presents sent by friends.
We navigated the players' attention to several important concepts.For instance, Ching Ching was able to move around in different maps and complete missions by talking to nonplayer characters.Questions in these conversations were used to highlight important concepts (see Fig. 5).By exploring different places and objects and playing various mini-games (see Fig. 6), players could also practice the skills repeatedly.
The level system recorded the level and skills of the player.The record was shared among the player's friends who had also joined the game, establishing a leader board to create competition and motivation.Moreover, the game facilitated social support by encouraging interaction between players (e.g., sending gifts and greetings among friends).Players could also share their experiences and knowledge by publishing game information on their timelines and/or chatting through instant messages.

Game development
The client-side of the system was implemented using Adobe Flash.Facebook iFrame and Javascript SDK were used to retrieve a user's profile and social network data from the Facebook application.Flash ActionScript 3.0 handled the logic of the game and sent requests to the back end in response to user actions.Javascript was used for back-end programming, and MySQL database stored game data.The system was hosted on a computer server on our institution's network.

Instruments
A web-based questionnaire that assessed demographic information and feedback with closed and open-ended questions was self-administered after a 3-week study period.The questionnaire first queried participants on gender, age, and Facebook ID.The feedback questionnaire consisted of close-ended questions on a 7-point Likert scale (1=strongly disagree, 7=strongly agree) in five aspects: (a) perceived learning effectiveness, (b) user satisfaction, (c) usability, (d) fun, and (e) Internet self-efficacy.For (a), (b), (c), and (e), the questions were modified from existing scales of learning and understanding (Foster et al., 2011), user satisfaction (Chin, Diehl, & Norman, 1988), usability (Chiu et al., 2005), and Internet self-efficacy (Eastin & LaRose, 2000), respectively.The questions for (d) were self-constructed.In the open-ended portion of the questionnaire, participants provided their comments and views of the online and social network DGBL.

Statistical methods
We used structural equation modeling (SEM) implemented in partial least squares (PLS) for data analyses.PLS is a second-generation regression method combining confirmatory factor analysis with linear regression, allowing us to simultaneously run measurement and structural models.The program used for the analysis was SmartPLS version 2.0.M3 (Ringle, Wende, & Will, 2005).

Measurement model
Table 1 summarizes the results of our measurement model, and Table 2 shows the intercorrelations with square roots of Average Variances Extracted (AVEs) on the diagonal.Our results demonstrated satisfactory item reliability, internal consistency, convergent validity, and discriminant validity for all constructs in accordance with the following conditions (Chin, 1998): (a) items' factor loadings on corresponding constructs were higher than 0.7, showing acceptable item reliability; (b) Cronbach's alpha and composite reliability were greater than 0.7, showing acceptable internal consistency; (c) the AVE of each construct was higher than 0.5, indicating satisfactory convergent validity; and (d) the square root of the AVE of each construct was higher than its intercorrelation with other constructs, providing evidence for adequate discriminant validity.

Structural model
We assessed the model's explanatory power via the R2 of endogenous constructs (Chin, 1998).As shown in Fig. 7, the model accounted for 14% of the variance in U, 35% of the variance in F, 41% of the variance in perceived learning effectiveness, and 72% of the variance in user satisfaction.The research hypotheses were tested by examining the size and significance of structural paths in the PLS analysis.Six of the research hypotheses were supported.Fun was a significant antecedent of both perceived learning effectiveness and user satisfaction, which supported H4a and H4b.Internet self-efficacy, however, was significantly related to perceived learning effectiveness, but not user satisfaction.Thus, H1a was supported, whereas H1b was not.In contrast, usability was significantly associated with user satisfaction, but not perceived learning effectiveness; H2a was not supported, but H2b was supported.Moreover, the direct relationship between Internet self-efficacy and usability was significant, supporting H3.The direct relationship between usability and fun was also significant, supporting H5.In addition, two factors, age and gender, were introduced as controls on perceived learning effectiveness and user satisfaction but were not significant.

Feedback
We collected qualitative comments through the questionnaire.The users' responses on the Facebook game were generally positive, with participants reporting that it was an innovative social network game.The game was also interesting for those who wanted to know more about mental health and depression.They welcomed the transformation from the standard form of education and agreed that teaching is no longer unilateral but learner-centered and interactive.Furthermore, they viewed the platform of education as gradually moving away from traditional classrooms to a more familiar social networking platform, Facebook, and found our approach modern and attractive.The participants also commented that the game was easy to share with new users via information posts and game updates on their personal Facebook pages.The participants were willing to share the game with other people, such as their friends and family members, since they considered it to be a good way to promote mental health awareness.
Since Facebook is popular, checking updates and using its applications have become a habit for many people.Most participants did not report any difficulty accessing the game and reported that playing it was very convenient.Facebook's personalization characteristics allowed us to automatically save players' game progress, allowing them to become more fully immersed in the game and learn more efficiently.
We also received negative comments.A few participants suggested that the proportion of interactive elements in the game should be increased since social networks provide a well-developed platform to implement fascinating social games.However, since this study did not primarily focus on social elements, we submit that social interaction in the game might not have been sufficient.Besides, Facebook inherently provides a highly interactive environment to users, which is much more important than the social elements in the game.For instance, players could discuss subject matters on their Facebook timelines and on our Facebook page.These comments will be instrumental to future development efforts.

Implications of the research for theory
Perceived learning effectiveness was an important measure of knowledge enhancement and an indicator of participants' attitudes toward educational usage of Facebook games.Its mean score was 5.22 out of 7 (SD=1.17;see Table 1), revealing that participants found it easy to learn about the material through the Facebook game with proper instructional design.The average score of user satisfaction not only demonstrated a general acceptance of DGBL in self-directed learning but also indicated that participants were satisfied with the learning process in the Facebook game.These findings may suggest participants' willingness to engage in continuous learning through Facebook games.
Fun is a useful concept in game design and education; however, it is less discussed in literature (Koster, 2010).Fun is the only factor positively affecting both learning effectiveness and user satisfaction (H4a and H4b).Previous studies have only examined how playfulness and enjoyment influence the intention to use systems (Venkatesh, 2000).Our findings provide insight into how educational Facebook games facilitate a comprehensive e-learning experience.Fun not only reveals the perception after experiencing the learning process in the Facebook game but also motivates users to achieve better learning outcomes.This paper provides an exploratory study of the associations between fun and other factors and raises the need for understanding fun in education on Facebook.
High Internet self-efficacy scores (M=4.70,SD=1.12) revealed participants' high capacity of control and management of their computers, effectively facilitating the learning process on their own.Although Internet self-efficacy has been indicative of search performance in some studies involving computer-mediated learning environments (Tsai & Tsai, 2003), it may also lead to high levels of learning in the Facebook game.Internet self-efficacy was shown to be directly and indirectly related to perceived learning effectiveness (H1a and H3).The result implies that the learning process may not only be restricted to the Facebook game; participants could learn and absorb information outside the game by searching for useful information to complete the game.Through this search process, players could strengthen their understanding and knowledge.Thus, DGBL should not only restrict learning inside the game but also strive for learning to occur outside the game.

Implications of the research for practice
Usability is considered to be one of the important characteristics of software and educational tools.A user-friendly system can provide easy controls and operations so novice users can learn how to operate the Facebook application easily.Clear organization, structure, and content representation also allow players to find useful information and clues to easily complete tasks in the Facebook application.Therefore, playing such a Facebook game requires minimal effort and hence facilitates the learning process.However, usability enhanced user satisfaction (H2b) but not learning effectiveness.If players find the game less exciting and challenging after completing some tasks, they may not attend to the game context and learning materials.They could simply ignore useful information and advance to the next plots, thus failing to efficiently learn the materials.Fun, however, enhanced both perceived learning effectiveness and user satisfaction (H4a and H4b), and usability significantly associated with fun (H5).
Designers and educators should incorporate innovative gaming elements to encourage enjoyment of learning.These findings have implications on future research and the multidisciplinary construction of large-scale games for e-learning and self-directed learning.

Limitations
There are some limitations in this study.First, we only focused on user experience and game factors.Other potentially influential factors, such as instructional design and technology, were not evaluated or included in the analyses.These factors may interact with game factors to affect e-learning outcomes.Besides, since the study had a small sample size and was only targeted at young adults, the results may not be generalized to other populations.A larger sample with different age groups is required for a more representative sample.Furthermore, the study was domain specific, a common issue in many prior studies.Findings might be different when attempting to teach different subjects; thus, further investigations are required.

Fig. 1
Fig. 1 shows our research model and hypotheses on Facebook's educational usage.The research model examined how Facebook game facilitates e-learning.We called the selfdirected educational Facebook game in the study "the Facebook game."The two outcome variables in the model were perceived learning effectiveness and user satisfaction in elearning outcomes after completing the Facebook game.Perceived learning effectiveness instead of actual learning effectiveness was used in the study, allowing generalization of our findings to other e-learning practices on Facebook applications.User satisfaction was assessed to evaluate acceptance of the Facebook game.Other variables in the model included Internet self-efficacy, usability, and fun.

Table 1
Results for the measurement model