An Investigation of Faculty’s Perceptions and Experiences when Transitioning to a New Learning Management System

A qualitative study using the cross-case analysis method investigated the experiences of faculty members as they transitioned from one Learning Management System (LMS) to another. The study has several implications for administrators, researchers, instructional designers, trainers, and educators on how to implement a new instructional technology and to facilitate transitioning efforts. Based on the findings of this study, we have identified several contributing factors to a successful transition to and implementation of a new LMS at the institutional level, including (a) the critical importance of ongoing systemic support at different levels and in different formats, (b) the demands for pedagogical support in addition to technological assistance, and (c) the requirements for customized support to address discipline-specific challenges. Lastly, we discuss why faculty should be empowered to join in decision-making processes regarding community-wide technology innovations.


Introduction
Adopting innovative technology and pedagogy for online learning has become commonplace in higher education (e.g., Bonk & Zhang, 2008;Zhang, 2008). Most colleges and universities have adopted some kind of Learning Management System (LMS) (Anderson & Jackson, 2000;Falvo & Johnson, 2007;Narwani & Arif, 2008). Increasing numbers of universities utilize LMSs for online or blended learning. The rapid rate of LMS adoption brought by emerging online technologies has led researchers to look beyond the adoption stage and adopter-based theories (Rogers, 2003;Surry & Fargquhar, 1997) to search for empirical guidelines and models focusing on implementation (Black, Beck, Dawson, Jinks, & DiPietro, 2007;Rubin, 2007).
Even as each adoption and diffusion process presents challenges (de Boer & Collies, 2001), there are many specific issues confronted by faculty that need to be addressed, since faculty members are very important stakeholders in the implementation process (Olcott & Wright, 1995;. For example, faculty are faced with the challenges of changing to a new LMS based upon adminstrative decisions, with the difficulties of converting a course from one LMS to the other, and with the need for technological and pedagogical support. Broadly, these represent social context issues that are by and large ignored during the adoption of a product. In addition, researchers have begun to emphasize the importance of user involvement in the design and diffusion processes of human learning systems (e.g., Carr, 1997;Carr-chellman, 2006;Carr-chellman & Savoy, 2003). Therefore, a thorough investigation of change and transition processes at the individual level may reveal user preferences that may help improve the utilization of instructional technologies in education and learning.
Attempts have been made to study individuals' perceptions and experiences about implementing technological innovations. For instance, Kurtz, Beaudoin, and Sage (2004) studied faculty members' perceptions about their novel and specialized roles as integrators of online teaching within the broader campus-based teaching context. Further, Ensminger (2002, 2003) sought to investigate individuals' measure of perceived importance of Ely's (1990Ely's ( , 1999 eight conditions that facilitate the implementation of educational technology innovations. Based on the results of the work, Surry and collegue subsequently created the Implementation Profile Inventory ). Ensminger, Surry, Potter and Wright (2004) also continued along this line of research by analysing the survey responses of 179 individuals from a broad spectrum of occupations.
Although useful, the above studies share a quantitative methodological preference for assessing the perceptions and experiences of various audiences, including faculty. Furthermore, while these studies explored the adoption and diffusion of new innovations in contexts where none existed before, they did not explore explicitly how individual faculty members switched to a technology innovation that was intended to replace one already in use for some time. A qualitative study focusing on faculty members' perceptions and experiences in this context may inform researchers and developers, as well as institutions, on how best to take advantage of the innovative technology of LMSs.

Purpose
The change from a widely used LMS to another at the universitly level is likely to have systemic impacts (Ely, 1999;Rogers, 2003), not only at the organizational, but also at the individual level (Reigeluth & Garfinkle, 1994;Rogers, 2003Rogers, , 1995Fullan, 1994;Holman & Devane, 1999). The faculty member's past experiences and institutionalized knowledge of the previous LMS may either facilitate or impede the transition to a new LMS. The purpose of this study was to understand, from the perspective of the faculty in one specific institution, how individuals transition to a new LMS from a pre-existing one. Through cross-case analysis of faculty interviews, we were able to gain a deeper understanding of the transitional processes involved. The following questions guided the inquiry of this study:  What are faculty's perceptions and experiences of a new LMS as they make the transition from an existing LMS?
 How do the user support systems meet the individual and group needs during the transitional period?

Context and Participants
A public university in the mid-west USA had been using Blackboard and WebCT in the past few years, but was recently switched to Desire2Learn(D2L) due to the administrative decision. Most faculty members had some experience teaching with a LMS. At the beginning of the transition, the institution provided different levels of support through training workshops, help desks, and online communities. College-level support varied: In some colleges instructional design consulations were available; in some others, there was only technical support. Six faculty members (1 male and 5 female) representing different disciplines, including liberal arts, science, and education were invited to participate in a research project. They all accepted our invitation to participate in this study. The participants' names reported in this article are all psudonyms.

Data Collection
A major data source for this research was interviews. The project interviews involved digitally recording participants' descriptions of their experiences centering the implemention of the new LMS. The participants were asked to sign consent forms and check the box indicating their agreement to be recorded. We chose to use a standard set of questions as an essential criterion for reliablity checking in our interviews, focusing on (a) the participants' experiences with the transition to a new LMS, (b) feature comparisons between the two systems, (c) the participants' perceived effectiveness of the LMSs in improving learning and instruction, (d) the systems' contraints and/or improvements, and (d) the participants' perceptions of the impact of the transition and their perceptions of institutional support. Finally, we wanted to ensure that any potential similarites in respondents' answers reflected variables external of the actual interview events (Silverman, 2001). Thus, the interviewers only deviated from the protocols when clarifications were necessary. Interviews varied from 30-45 minutes in length.

Data Analysis
The researchers listened to each digitally audiotaped interview several times to become familiar with the dialogues before performing open coding. The data was disaggregated into fragments by extracting significant sentences, phrases, and words from the participants' statements. After eliminating any repeated information, the fragments were prepared into a preliminary list of meaningful formulations, which are displayed in Table  1.
The preliminary meaningful statements were then re-aggregated to form a deeper level of meanings. We arrived at the meanings by rereading and reflecting on the participants' original statements. The aggregate data were organized using a series of thematic headings that emerged from the data (Atkinson, 1990(Atkinson, , 1992. The aggregate themes represent meaningful descriptions common to all participants and are presented in Table 2. We then searched for clusters in the themes and drew diagrams and charts to help visualize and refine the relationships among the themes. Further, we re-examined the original interview data to see if there was anything that was not captured by the thematic headings, but also, to see if the themes captured something that was not in the original data. Figure 1 (see the Discussion section) represents how we organized the clusters of themes.

Role of the Researchers
Two of the three researchers in this study conducted the interviews. One of the researchers was an instructional design consultant assisting faculty with the new LMS at the institution at that time. This researcher, therefore, had an insider's perspective. The other researcher who conducted the interviews was a faculty member at the same university, and thus, was familiar with the cultural context of the institution. The third researcher, who was from another institution, had extensive research experience in using emerging tools to support online learning and instruction. This researcher contributed in developing interview protocols and subsequently assisting with data analysis. Collectively, the three researhers possessed both emic and etic (experience-near and experience-distant) perspectives (Geertz, 1976;1983) that were essential in strengthening the validation of the research process in terms of data collection and analysis.

Research Question 1
What are faculty's perceptions and experiences of a new LMS as they make the transition from an existing LMS? Responding to this question, we report findings on (a) the overal impact of the transition on the institution and individual faculty members, and (b) individuals' technological experiences with the new LMS in comparison with the prior LMS. These findings, organized and represented by the thematic headings as found below, were the results of the second-level data analysis in formulating meanings of significant statements, as shown by Table 2. Each of the following thematic headings was supported by meaningful preliminary statements as found in Table 1, which were aggregated to form a deeper level of meanings.

Overall effects of the transition
Overall, the transition was reportedly very smooth. Jan said, "Oh yeah, it was easy." The other interviewees said that they "haven't heard any complaints," and "instructors are not talking about it, and students had no problems with it (D2L)." They felt that the transition was handled pretty well at the institutional level. Workshops were offered in various colleges, while help was also available via the university's Information Technology (IT) support department. Jan attended a workshop about the new LMS, which she said was useful, but she indicated that she had learned more by calling IT Help whenever she had a question. She said that she called the IT Help desk a lot, which had been tremendously helpful in facilitating her transition to the new system.
An inquiry of the transitional effects may reveal the causes of the smooth transition by the institution. Importantly, the participants' experiences varied in the beginning stages. While switching to the LMSs did not bother some participants, others were worrisome. Helen described her first feelings as "panic," "terrible," and she wished that "she could drag her feet" through this transition. Winter also expressed the frustration she had experienced in the first semester when the new LMS was just introduced and implemented. It was a significant transitional process for her, which was making a big impact on her instructional life.
A deeper probe revealed that individuals' prior experiences determined their initial responses to the implementation. For example, Ron, Amanda, Caissie, and Jan had all used more than one LMS before, and their prior knowledge and experience easily transferred to the learning and use of a new LMS. By comparison, Helen had been learning and using Blackboard only for the past two semesters, and she had just become "comfortable" with Blackboard when the change occurred. However, as Helen learned more about the features of the new LMS, she became more comfortable with the new system over time. She said that her learning experience with Blackboard was like "a frame of reference" to "reorient" her to the new LMS. Winter was more concerned about the amount of time she had to spend transferring her course files from one system to the other. Thus, we infer that the transition at the institutional level was highly related to the individual experiences of faculty in the transition. To a large extent, the impact of the transition on individuals depended on individuals' prior knowledge and experiences and the amount of time they had to commit.
Additionally of interest, while some faculty members seemed to have experienced a brief learning curve at the beginning, these participants moved beyond the early learning stages much sooner than they did with the previous LMS. Both Helen and Winter indicated that their training and troubleshooting skills in Blackboard transferred to their learning experiences with D2L. However, institutional support may also help individuals to accelerate their transition process. For example, the training sessions and workshops at both college and university levels helped faculty to overcome their initial overwhelming feelings, knowing that the help was readily available.

Prior experience
We observed that the participants, when they first started using the new LMS, made comparisons between the new and the old system, constantly searching for the new LMS's "ideal" features in comparison with the old LMS. If their prior experience with some features in the old system had been frustrating, or if their expectations were not met by the old system, they would hope to find their ideal features in the new system. However, if their experiences with some features of the prior system had been positive, they would hope that the similar features could be found in the new system.
From the interview data, it seemed obvious that when the participants were asked to described their "likes" or "dislikes" with the new system, they would make reference to their past experiences with some particular features they liked or disliked in the old LMS regarding their instructional needs. For example, as a Spanish professor, Caissie liked the new LMS because it was able to handle Spanish letters, accents, and special characters, which were beyond the capability of the old LMS. She also liked some grading features that were absent in the prior LMS. Jan also echoed the same preference about the features of the grade book in the new LMS.

Interface features
The participants were clear about the main interface features they would like to have in a typical LMS. The most popular of these feature descriptors of the new LMS included easy, intuitive, visual, organized, transparent, flexible, and dynamic. On the other hand, the participants also registered complaints about the LMS being inconvenient. For instance, the emailing system was not self-contained in the new LMS, that is, all the emails sent from this new LMS would go to individual instructors' mailboxes, which could result in an instructor receiving massive volume of email messages from students sent from their course site.
User preferences could influence the sustaining use of various LMS features and affected participants' decision-making in instructional design and delivery. Most of the faculty expected that a LMS should be intuitive and easy to navigate. If these expectations were not met, they would probably use the system less or quit using it, eventually. For example, Amanda indicated her dissatisfaction with the absence of a certain feature (that is, embedding a hyperlink anywhere in the text), which she considered a big limitation of the system. Regardless of the accuracy of her claim, this perception had led her to seek alternative solutions outside the LMS.

Visual organization
Several participants commented on the visual organization feature of D2L, which not only helped the instructors to organize instructional materials, but also served as an advanced organizer for students. Both Caissie and Winter indicated that in D2L everything fell into one place, which made it easy for instructors to perform teaching activities and for students to "pull off the materials" from one place and "make connections." In addition, Winter said that the visual and "centrally located" features of the new LMS enabled and strengthened student group activities. From this perspective, we infer that the instructors believed that a well-designed LMS feature could function as a cognitive tool to facilitate students' critical thinking.

Options vs. control
Several participants appreciated the "flexibility" and "options" (e.g., widgets, links, etc.) afforded by the new LMS, which enabled them to customize the course design, experiment with different instructional techniques, and further extend their "creativity." Helen said that she was usually very creative with classroom teaching activities, and now she could also be creative online thanks to the new LMS. Similarly, Winter mentioned that the new LMS had allowed easy uploading and sharing of multimedia files, which reduced the frequency of students navigating away from the course site.
However, we observed a tricky balance between flexibility and control. While most of the individuals we interviewed liked to have flexibility in a LMS with various options, they also simultaneously wanted to have good control over the system. As Winter described, "I like the control." Blackboard provided her "just that kind of control" since it was "straightforward" with fewer options. We noted from the interviews that while D2L provided more options and freedom for customization, it could seem overwhelming to some faculty members. However, as the instructors gained more competence and confidence with the new system they began to exhibit greater appreciation for the perceived freedom. Moreover, they began to reconsider how they could better design their courses with the options available. This is just as Winter expressed: "I had to rethink what is the best way to do things" for her course design and student activities. Winter went on to explain, "It all depends on how I used it (the system). Even though the system works for me, it depends on me to make the course interactive." However, some participants did not perceive the meaningfulness of those options or opportunities. For example, Ron and Amanda reported only using some of the basic functions.
We concluded that although the system was a precursor for good instructional design, arguably, it really depended on the instructor or the designer to make the system fully functional, interactive, and beneficial. To that end, faculty must first see the usefulness of the tools so that they would be willing to use them. However, the paradox was that to be able to perceive the usefulness of the tools, one must first of all began to use them. For example, when asked about her ideal LMS features, Jan said, "There are a lot of features I am not using. But I think it [the LMS] is probably more ideal if I use it more." Therefore, technology affordances, knowledge, intentional use, and perceived usefulness were complexly intertwined.

Research Question 2
How do the user support systems meet the individual and group needs during the transitional period? Through data analysis, both preliminary and secondary formulating meanings of significant statements (refer to Table 1 and Table 2), we identified several levels and types of user support systems and their impacts on the LMS transitions. Two main types of support were technological support and pedagogical support. All the participants acknowledged and commended the availability of the technical support and assistance at the university level, such as the institution's task force, the instructional technology (IT) group, workshops and Helps Desk. In addition, workshops were also offered at the college level. At some colleges, such as College of Arts and Sciences (CAS), there was an instructional design team helping faculty navigate the transition.
However, the data showed college-wide differences in the extent and the nature of the LMS use and the support received by individuals. As mentioned, College of Arts and Sciences (CAS) had an ongoing support system sustaining the implementation of the LMS. According to an instructional design consultant, the CAS not only provided faculty with training in technology use, but also in pedagogy and instructional design. In addition, there was also an online discussion forum, where many innovative D2L ideas were exchanged and technical problems were discussed and solved in the virtual support community. As a result, the faculty members at CAS, like Winter, Helen, and Cassie, were able to describe and discuss the detailed functions and their instructional impacts. Pedagogically, the participants from the CAS also seemed more ready to incorporate media files into the system (e.g., Caissie) and more willing to experiment with group work online (e.g., Helen and Winter). By comparison, the pedagogical support seemed to be largely overlooked in some other colleges. For example, Ron admitted that other than the basic features, he did not experiment with the advanced features because he did not know what they could be used for.

Discussions
There were major themes that emerged from the data analysis, which were reported in the Findings section. Those themes helped us to identify the interrelationships among different factors and build a conceptual model illustrating those complex interrelationships. Figure 1 provides one possible representation of the interrelationships between a LMS (i.e., options, interface design, and features), user characteristics (i.e., control, confidence, competence, perceptions, knowledge, and prior knowledge), and institutional support (i.e., technological support and pedagogical support).
Let us begin with the characteristics of the LMS. According to our participants, an easy interface design was one of the "must-haves" of an "ideal" LMS. One participant used the term "transparent," indicating that the ideal interface must be somewhat visual and intuitive. An intuitive interface allowed users to clearly identify the features of the LMS. One important aspect to facilitate the transition from one technology to the next was the ability to easily identify the features of both systems. The faculty members were constantly searching for an ideal LMS by comparing the features of their current LMS with the previous one. Later we will discuss the reasons why faculty tended to take this "features approach." For now, we must point out that the most common feature in the new LMS was the availability of a wide range of user options, which led to users' perceptions of control.
Upon further investigation of the notion of control, we observed differences in the two LMSs in terms of the number of options that were available for users to command. The prior LMS was relatively easy to use, did not provide many options, and thus the faculty members had little worry over customization. The new LMS provided more options and enabled perceptions of freedom to customize, but also seemed overwhelming to use at first. The faculty members seemed aware of the tradeoffs and chose control over ease, even with the initial discomfort of learning the new system. We therefore wanted to explore the nature of the perceived control.
We discovered that perceptions of control were related to organization and management of the learning environment or cognitive "space." For example, one of the participants stressed the importance of being able to control the appearance and positioning of pictures. When asked to explain the importance of this control, Helen replied, We infer that if we treat a LMS as a "cognitive tool (Jonassen & Carr, 2000), control "space" impacts teaching strategies, which in turn leads to more learning activities. It allows a teacher to rethink about the course design. Further, control can be examined in two ways: one way relates to the technology itself, that is, technology affordances; and the other relates to user characteristics. For instructors, the technology provides an opportunity to design activities and rethink course design. For students, it provides a single, centralized "space" or learning environment for everything, and enhances learning and group activities. Here we draw upon a recent argument that new developments in information technology have created a paradigm shift away from what is traditionally viewed as "teaching" to what is now commonly accepted as "learning in computer educational environments" (Okamoto, Christea, & Kayama, 2001). What this means is that the role of the teacher has changed to that of a manager or facilitator of the new forms of pedagogical support afforded by the computer learning environment (O'Donoghue, Singh, & Dorward, 2001), and correspondently students' roles have also changed to that of managers of their own learning as they become less dependent on their teachers. The way the new LMS allows users to actively participate in the improvement of the system design may be described as a new culture since educators act as change agents who are involved in the evolution of educational technology (Ely, 1999). We must therefore seek to explore and nurture this new culture.
The other way to examine perceived control relates to the characteristics of the faculty. The process, by which faculty members in this study evaluated the two LMSs, opened a new window into faculty members' ideas about the notion of control. The faculty members have evaluated the LMSs on the basis of their perceptions of their own confidence and competence. Therefore, we deduce that confidence and competence are two very important interrelated and co-defined elements of participants' perceptions of control (refer to Figure 1).
In terms of a relationship, it may be inferred that while the perceived control afforded by the learning environment may influence personal confidence in using the tools, alternatively, the learning environment may also, in fact, enable perceptions of control. Further, while confidence and control are related, confidence is also linked to users' competence. We can also see that the users' competence, and therefore, their perceptions of confidence and control, may be related to their perceptions of the utility of the LMS, as well as their knowledge in using a LMS (including all their prior knowledge). Perceptions of utility are tied to users' prior experiences with an LMS and their expectations that the features of the LMS will enable them to solve problems (refer to Figure 1).
While individuals engage technologies intending to solve particular problems, the technologies or tools themselves mediate how individuals interact with their own realities. This brings to mind a compelling case for the relationship between pedagogy and technology evolution. Pahl (2003) draws on Activity Theory (Nardi, 1997) as a conceptual framework of computer-supported interactions. Pahl (2003) explains that tools or technologies reflect the experiences of other individuals who have been trying to solve similar problems. He further explains that designers' accumulated experiences represent the structural and behavioural properties of the technology. In this sense, we can think of a LMS as a tool that mediates learners' interaction with reality, which is the domain content and the pedagogy. Thus, we see instructional design as the essential activity that can combine technology and pedagogy.
The faculty members in our study clearly understood their dual roles as instructional designers and teachers/facilitators, and the way these roles are implicated in the design and management of online courses. We believe this is the reason the faculty members were drawn to the "features approach" in evaluating the new system. The features represent other people's (developers and testers) efforts in solving meaningful educational problems. The features of the technology can facilitate active learning, thereby creating access to meaningful representations of reality for students, when the right pedagogical approaches are utilized.
In their traditional roles as instructional designers, most of them had been trained to use the Analysis, Design, Development, Implementation, and Evaluation (ADDIE) model or some variant. The faculty understood and articulated that although the model had powerful guidelines for the Analysis, Design, and Development of instructional products, the model was inadequate in supporting the Implementation and Evaluation phases. This insufficiency becomes underscored when faculty members were left helpless to design and develop their courses and to prepare assessments to evaluate the effectiveness of their instruction. Due to the complexities of virtual learning environments, if there was no external support, the hopes of successful implementation and evaluation would wither.
This brings us to our final major theme, support. In this study, we saw how individuals engaged with the LMS differently based on the support systems available to them. Those who had technical as well as pedagogical support (including communitybased support) used more features and created more variety of useful activities for their students. Therefore, we recommend instructional design support, especially in the areas of implementation and evaluation, to be administered simultaneously with technological support at all stages when transiting to a new LMS.

Implications
This study has several implications for researchers, administrators, instructional designers, trainers, and educators on the implementation of, or change to, a new instructional technology. Several factors were identified as desired ones for successful transition to a new LMS. First of all, ongoing systemic support should be made available at different levels (e.g., individual, departmental, college, and university, etc.), and in different formats (e.g., one-on-one, group training, just-in-time training, real-time support, communities of practice, communities of learning, etc.). Secondly, it is important to understand that at the implementation stage, equal focus should be placed on pedagogical support as on technological assistance. Thirdly, understanding the specific needs for various subject areas or domains is essential in helping faculty fully take advantage of the learning systems. Lastly, faculty should be empowered with a sense of ownership in the change process by participating in the decision-making processes regarding communitywide technology innovations.
The relatively small number of participants is a limitation of this study. Similar studies with more participants in other institutions would help further corroborate, and refine our knowledge in this area, and extend the scope of our inquiry. For example, data collected from other members of the university community, such as students, administrators and technology support staff may help builder a fuller and deeper understanding of the dynamics of such a transition. In addition, while this paper focused mainly on interview data, future studies may also integrate archived document analysis as well as examining artifacts (e.g., instructor's actual D2L course sites, user notes, etc.).
Findings of this study suggest that the implementation of a new LMS at a university would require systemic support at various levels and in multiple formats. Strategies found helpful in this study may be applicable to other institutions in similar transitions to provide knowledge, skills, and sustainable motivation for a smooth transition (Dawson & Jones, 2003). Despite the enormous complexity and highly diverse culture (by individual, by discipline, by college, etc.) in the higher educational institution we investigated, we believe we have discovered the factors that contribute to a successful transition to a new LMS at that institution. These include (a) the critical importance of ongoing systemic support at different levels and in different formats, (b) the demands for pedagogical support in addition to technological assistance, and (c) the requirements for customized support to address discipline-specific challenges. More importantly, faculty were empowered to join in decision-making processes regarding the community-wide technology innovation.