A quantitative analysis of learning object repositories as knowledge management systems

Learning Object Repositories (LORs) are a core element of the Opening up Education movement around the word. Despite, the wide efforts and investments in this topic, still most of the existing LORs are designed mainly as digital libraries that facilitate discovery and provide open access to educational resources in the form of Learning Objects (LOs). In that way, LORs include limited functionalities of Knowledge Management Systems (KMSs) for organizing and sharing educational communities’ explicit and tacit knowledge around the use of these educational resources. In our previous work, an initial study of examining LORs as KMSs has been performed and a master list of 21 essential LORs’ functionalities has been proposed that could address the issue of organizing and sharing educational communities’ knowledge. In this paper, we present a quantitative analysis of the functionalities of forty-nine (49) major LORs, so as (a) to measure the adoption level of the LORs’ functionalities master list and (b) to identify whether this level influences LORs’ growth as indicated by the development over time of the number of the LOs and the number of registered users that these LORs include.


Introduction
Opening up education is a global movement that aims to facilitate open and flexible learning by exploring the potential of ICT to improve education and training (Conole, 2013;Iiyoshi & Kumar, 2008).Open educational resources (OERs) constitute a significant element of the opening up education movement (The William and Flora Hewlett Foundation, 2013;UNESCO, 2012).Within this context several OER initiatives have been developed worldwide by large organizations/institutions such as UNESCO OER Community1 , Open Education Europa2 , Carnegie Mellon Open Learning Initiative3 , MIT's OpenCourseWare4 (OCW), Stanford's iTunes5 and Rice University's Connexions6 , or by communities (or consortia) such as MERLOT 7 and OER Commons8 (Ehlers, 2011;Walsh, 2010).The main aim of such initiatives is to support the process of organizing, classifying, storing and sharing OERs in the form of Learning Objects (LOs) and their associated metadata in web-based repositories which are referred to as Learning Object Repositories (LORs) (McGreal, 2008).
As a result, a variety of LORs are currently operating online, facilitating targeted end users (mainly, teachers and learners) to have access to numerous collections of LOs (Ehlers, 2011).However as discussed in Sampson and Zervas (2013a), despite the wide efforts and investments in this area, most of the existing LORs are being designed mainly as digital libraries rather than knowledge management systems.As a result, they mainly provide functionalities for the organization and sharing of educational communities' explicit knowledge (typically depicted in the LOs constructed by teachers and/or instructional designers), but they come short in functionalities for the organization and sharing of educational communities' tacit knowledge (typically depicted in teachers' and learners' experiences and interactions using LOs available in LORs).This is an important shortcoming, since both aforementioned knowledge types are very important to be managed, shared and reused effectively among educational community members (McLaughlin & Talbert, 2006).This could also be a potential obstacle for the LORs' future use and growth rate, with growth in number of LOs and growth in number of registered users being key indicators in relevant studies (Ochoa & Duval, 2009).
In previous work, reported in Sampson and Zervas (2013a) an initial study of examining LORs as Knowledge Management Systems (KMSs) has been performed.Deriving from this process, a master list of essential LORs' functionalities (MLF) for addressing the issue of organizing and sharing both types of educational communities' knowledge, has been proposed.Extending this work, the main goal of this paper is to provide empirical answers to the following questions:


What is the adoption level of the LORs' functionalities master list by existing major LORs?  How does the adoption level of the LORs' functionalities master list influence LORs' growth?To answer these questions, data from 49 major LORs were collected and analyzed.The results of this process can assist us in gaining insight on the design of existing LORs and to what extent can be considered as KMSs.Moreover, we can identify the level of influence that LORs' design has on their growth.Finally, we can identify potential principles that can drive the development of future LORs towards addressing the issue of organizing and sharing educational communities' explicit and tacit knowledge.
The paper is organized as follows: Following this introduction, in section 2 we provide an overview of the different types of educational knowledge generated and shared within web-based educational communities of practice and discuss how these knowledge types can be facilitated by a master list of LORs' functionalities as identified in our previous works.In section 3, we present and discuss related works from the literature that deal with quantitative analysis of LORs, in order to identify useful insights about their popular features and growth patterns.In section 4, we present the method of quantitative analysis of 49 major LORs from a knowledge management perspective and we discuss the results of our study.Finally, we present our concluding suggestions.

Background: Management of educational communities knowledge in learning object repositories
Communities of practice (CoP) initially proposed by Lave and Wenger (1991) as: "a group of people who share an interest, a craft, and/or a profession.It can evolve naturally because of the member's common interest in a particular domain or area, or it can be created specifically with the objective of gaining knowledge related to their area of interest", are now well supported by web-technologies (Hara, Shachaf, & Stoerger, 2009).This has led to an increased interest for exploiting CoPs in the field of education and training.As a result, educational communities of practice have been developed focusing on generating, sharing and reusing different types of educational knowledge (McLaughlin & Talbert, 2006).These different types of educational knowledge can be divided into two types, as shown in Table 1.

Table 1
Types of educational communities knowledge (Sampson & Zervas, 2013b) Types of Educational Communities Knowledge Definition

Knowledge for educational practice
This is formal knowledge depicted in the LOs that are constructed by teachers and/or instructional designers of an educational community and they can be used to enhance teachers' day-to-day educational practice.This type of knowledge can be considered as explicit, since it can be codified, stored and articulated using certain media

Knowledge of educational practice
This type of knowledge is constructed: (a) by teachers based on their experiences about their learners' learning and evidence of their progress in relation to given LOs, (b) by learners based on their experiences about the use of given LOs provided by their teachers, and (c) by teachers-students interactions with these LOs.This type of knowledge can be considered as tacit, since it needs special effort to be codified and transferred As a result, in order to facilitate the different types of educational knowledge that need to be organized and shared within educational communities, in our previous work reported in Sampson and Zervas (2013a), we have studied LORs as knowledge management systems.More specifically, an initial study of existing LORs from the KMS perspective has been performed and a master list of essential functionalities has been proposed.The latter could address the issue of organizing and sharing both types of educational communities' knowledge, as shown in Table 2.

Store
This functionality enables LORs' end users to store in the LOR the metadata descriptions of their LOs, so as to be able to reference them with unique URLs for future

View
This functionality enables LORs' end users to view in details the metadata descriptions of LOs, so as to be able to decide whether to use or not a specific LO

Download
This functionality enables LORs' end users to download the metadata descriptions of LOs in XML format conformant with IEEE LOM Standard, so as to further process them with appropriate educational metadata authoring tools and upload them back to the same LOR or to another LOR

Validate
This functionality is used for validating the appropriateness and the quality of the metadata descriptions provided for the LOs by their authors and in many LORs this functionality is available to a limited number of back-end users (namely, metadata experts), who undertake the task to ensure the quality of metadata descriptions Social Tagging This functionality enables LORs' end users to characterize LOs by adding tags to them.

Personal Accounts
This functionality enables LORs' end users to create and manage their own personal accounts by completing their personal information and preferences.User accounts include also information about: (a) the LOs that a user has contributed to the LOR, (b) the LOs that the user has bookmarked and (c) the ratings/comments and tags that the user has provided to the different LOs of a LOR Forums This functionality enables users to communicate and exchange ideas in an asynchronous way about the use of LOs that are stored in a LOR Wikis This functionality facilitates users to create wikis and share information about their experiences with the LOs that are stored in a LOR

RSS Feeds
This functionality enables users to be informed via RSS readers about new LOs, which are added to the LOR without visiting the LOR

Blogs
This functionality enables LORs' end-users to build and maintain their own blogs for publishing their opinions about LOs stored in LORs and receiving comments from other end-users about their reflections

Social Networks
This functionality enables LORs' end-users to build online social networks based on the LOs that they are offering to the LORs, so as to share their common interests.

Related studies: Quantitative analysis of LORs
In this section, we provide an overview of existing studies that focus on quantitative analysis of LORs.In these studies, different LORs have been quantitatively analyzed, based on general characteristics such as metadata standard used, language, end users, quality control, as well as their growth rate.
McGreal ( 2008) has conducted a comprehensive survey of existing LORs and classified them in various typologies.The results of this survey revealed principal functionalities of LORs that are commonly used in existing implementations of LORs.More specifically, it has been identified that "search/browse LOs", "view LOs", "download LOs", "store LOs" and "download LOs metadata" were principal functionalities in the studied LORs.Ochoa and Duval (2008) has conducted a detailed quantitative study of the process of publication of LOs in LORs.The study focused on basic characteristics of the LORs' growth, namely LOs and registered users' growth over time.The main findings from this study were that the amount of LOs is distributed among LORs according to a power law, the LORs mostly grow linearly, and the amount of LOs published by each contributor follows heavy-tailed distributions.They have identified that all examined LORs had an initial stage of one to three years with low growth rate, whereas after this period, a more rapid expansion was observed as a result of the increased number of contributors of the LOR.Tzikopoulos, Manouselis, and Vuorikari (2009) have studied general characteristics of well-known LORs such as educational subject areas covered, metadata standard used, LOs availability in different languages, quality control, evaluation mechanisms and intellectual property management.This study provided an overview about LORs' current development status and popular features that they incorporate.More specifically, the majority of the studied LORs were cross-disciplinary, whereas a smaller, yet significant number were thematic LORs focusing on specific disciplines (e.g.mathematics, language learning, etc.).Additionally, the majority of the studied LORs were using standardized educational metadata for their LOs and they applied quality control processes for the LOs that are stored.
Finally, Ochoa (2011) has conducted a detailed quantitative study in order to measure and identify how learning objects are offered or published.The main findings from this study provided useful insights about the typical size of different types of LORs, as well as how different types of LORs grow over time.More specifically, it has been identified that the actual growth function for most LORs is linear and this is also applicable for even popular and active LORs.
As we can notice from the aforementioned studies, quantitative analysis of LORs can lead to useful insights about popular features that they incorporate, as well about their growth patterns.Nevertheless, none of the existing studies have been focused on possible factors that can affect LORs' growth.The research presented in this paper addresses this issue and aims to identify whether the adoption level of the master list of LORs' functionalities (presented in Table 2) can affect LORs' growth.

A quantitative analysis of LORs functionalities from the knowledge management perspective
In this section, we present a quantitative analysis of LOR functionalities from the knowledge management perspective.First, the method of analysis is outlined by presenting our sample, as well as describing the process followed for analyzing it.Then, the results are presented and finally the implications of our findings are outlined.

Sample
Our sample list was compiled from the following sources: (a) a list of LORs provided by the Wiki Educator (http://wikieducator.org/), (b) a list of LORs provided by OpenDiscoverySpace Project (http://www.opendiscoveryspace.eu/repositories), which is a major European Initiative aiming to build a federated infrastructure for a superrepository on top of these LORs and (c) a list of LORs provided by EdReNe (http://edrene.org/),which is an EU-funded thematic network aiming to bring together a network of LORs and stakeholders in education.Our full sample list is presented in Table 3.More precisely, Table 3 provides details about:


The subject domain that the LOs in each LOR target, namely (a) thematic LORs (that is, only one subject domain) and (b) cross-disciplinary LORs (that is, more than one subject domains).


The regional features of the community that each LOR targets, namely (a) national LORs, (b) European LORs and (c) international LORs.


The type of the LOR, namely (a) simple LORs and (b) federated LORs (which provide access to LOs from different LORs).


The total number of users and LOs that each LOR includes.


The age of each LOR, namely the years that each LOR has been operating online.

Process
For each LOR presented in Table 3, we studied which functionalities of Table 2 have been adopted in its implementation.Next, we estimated the average number of LOs and registered users per year.This has been calculated by dividing the number of LOs and the number of registered users with the LOR's age.Finally, we calculated Kendall's tau correlation coefficient between the adoption level of Table 2 functionalities and the average number of LOs and registered users per year.It should be noted that for the process of calculating the registered users related correlation coefficient, our sample was reduced to forty-three (43) LORs due to lack of data of registered users for six (6) LORs, as previously explained.

Adoption level of master list LORs' functionalities
Fig. 1 presents the adoption level of master list LORs' functionalities (MLF) for every LOR in our sample.The adoption level has been calculated for the functionalities of each of the three components identified in Table 2.
As we can notice from Fig. 1, none of the examined LORs incorporates all 21 MLF, listed in Table 2.Moreover, it should be mentioned that functionalities related to the LOs component are the most dominant to the examined LORs, whereas the functionalities related to the added value services component are limited.
Next, we calculated the number of occurrences of the MLF in our sample.This information is depicted in Fig. 2.
As we can notice from Fig. 2, "MLF #2 -Search" and "MLF #3 -Browse" both related to the LOs component are used by all examined LORs in our sample, whereas the "MLF #18 -Wikis" of the added value services component is used by only 2% of the examined LORs.As we can notice from Table 5, there are a number of statistically significant correlations between the variables, although the correlations are low.More specifically, there is a weak correlation (τ=0,21, p<0,05) between the adoption level of the LOs component's functionalities and the average LOs per year.Moreover, there is a weak correlation (τ=0,24, p<0,05) between the adoption level of the added value services component's functionalities and the average LOs per year.On the other hand, there is no significant correlation between the adoption level of the metadata component's functionalities and the average LOs per year.Based on these results, we can suggest that LOs component's functionalities and added value services component's functionalities can only marginally affect LOs growth in LORs.
Furthermore, based on the results of Table 5, there is a weak correlation (τ=0,20, p<0,05) between the adoption level of the LOs component's functionalities and the average registered users per year.Moreover, there is a weak correlation (τ=0,19, p<0,05) between the adoption level of the added value services component's functionalities and

Conclusions
In this paper we report on a quantitative analysis of the functionalities of a significant amount of LORs that are currently operating online.This analysis was based on a master list of 21 functionalities (MLF) that has been identified in our previous work and aims to identify the adoption level of MLF by existing major LORs.Moreover, the influence of the adoption level of these functionalities master list to the LORs' growth was studied.The results of our analysis provided us with indications that:


Current LORs' implementation adopts mainly functionalities that are related to the LOs component of the master list of functionalities, whereas functionalities related to the added value services component are limited.This provided us with evidence that current LORs are mainly developed for facilitating the storage and retrieval of LOs, whereas functionalities for facilitating interactions between teachers and learners when using LOs available in LORs are rarely supported.


Adoption level of the LOs component's functionalities and the added value services component's functionalities can only marginally affect LORs' growth.
On the other hand, adoption level of metadata component's functionalities does not affect LORs' growth.


Master list functionalities can be classified into four main categories (based on their occurrence frequency), namely core, essential, optional and rare functionalities.LORs growth can be weakly affected by utilizing the set of both core and essential functionalities and it can be moderately affected when the optional functionalities are also included.
The aforementioned indications could facilitate developers of LORs during the process of developing new LORs or enhancing existing LORs targeting to achieve higher growth rates of these LORs.

Table 2
Master list of LORs' functionalities from the knowledge management perspective LORs' end users to store in the LOR their LOs and/or links to external LOs, so as to be able to reference them with unique URLs for future use and sharing them with other users.
Mash-ups refer to web applications which present data acquired from different sources and combined in a way which delivers new functions or insights.This functionality enables LORs' end-users to perform federated searches and retrieve LOs from other LORs.

Table 3
List of selected LORs 1

Table 2 .
 Essential Functionalities, namely those that are used by 45% up to 85% of our sample LORs.This category includes six (6) functionalities only from the LOs and the Metadata components listed in Table2.In this section, we calculate the Kendall's tau correlation coefficient between the adoption level of MLF and the average number of LOs per year, as well as the average number of registered users per year for each LOR in our sample.We have selected to calculate Kendall's tau correlation coefficient because our data are non-normally distributed.The correlation coefficients have been calculated (a) per adoption level of each component's functionalities listed in Table2and (b) per adoption level of each classification category's functionalities resulted by occurrence frequency and presented in section 4.2.1.

Table 5
presents the calculated Kendall's tau correlation coefficient between the average number of LOs per year, as well as the average number of registered users per year and the adoption level of each component's functionalities listed in Table2.

Table 5
Kendall's tau correlation coefficient per adoption level of each component's functionalities N: Denotes our LOR sample