Information Quality Framework for e-Learning Systems

: Information quality frameworks are developed to measure the quality of information systems, generally from the designers’ viewpoint. The recent proliferation of e-services and e-learning particularly raises the need for a new quality framework in the context of e-learning systems. This paper proposes a new information quality framework, with 14 information quality attributes grouped in three quality dimensions: intrinsic, contextual representation and accessibility. This framework could be useful to e-learning systems designers, providers and users as it provides a comprehensive indication of the quality of information in such systems. We report results based on original questionnaire data and factor analysis supporting our conclusions.


Introduction
Today quality is considered a crucial issue for education in general, and for e-learning in particular.Currently there are two recognized challenges in e-learning: the demand for overall interoperability and the request for high quality.Moreover, quality cannot be expressed and set by a simple definition, since in itself quality is a very abstract notion.The specified context and the perspectives of users need to be taken into account when defining quality in e-learning.It is also essential to classify suitable criteria to address quality (Stracke, 2006).
In the literature, there is a wide interest in information quality provided by information systems in general.However taking into account that quality on the web is a complex concept and its measurement is expected to be multidimensional in nature (Aladwani & Palvia, 2002), the prime issue in evaluating the quality of any information system is identifying the criteria by which the quality is determined (Buyukozkan, Ruan, & Feyzioglu, 2007).The criteria are a result of the multidimensional and interdependent nature of quality in information systems, and are dependent on the objectives and the context of the system.This paper is part of a wider research project aiming to define metrics to determine the quality of the content provided by distributed learning materials, for integrating intelligent agent technologies as a means of gathering information for quality evaluation.
This paper focuses on concepts of information quality in the context of e-learning systems, particularly on identifying the key dimensions for information quality from the users' perspective in order to build a quality framework to measure the quality of the content provided by e-learning systems.It is essential to identify quality dimensions accurately as they provide the building blocks for further research into the quality of elearning information systems in general.Great attention has been given to ensure the accuracy of the diminutions defined in this paper.In our study, Wang & Strong's data quality framework (Wang & Strong, 1996) was extended and used as the reference point owing to its popularity and acceptance by the information systems quality community.
The rest of the paper is organized as follows.The next section reconsiders the meaning of e-learning and its definition including the concept of quality in e-learning systems.Section three reviews previous research related to information systems quality frameworks and proposes the first draft of the new framework.In section 4, we discuss our work to collect learners' opinion to identify information quality characteristics in elearning systems and the preliminary results.Data analysis and the revised framework format are presented in sections 5 and 6 respectively, followed by the conclusion and future work in the final section.

E-learning
The term e-learning is used in literature and commercial applications to describe many fields such as online learning, web-based training, distance learning, distributed learning, virtual learning, or technology-based training.During the last decade, e-learning was defined in literature in different ways.In general, most definitions for e-learning are used to express the exploitation of the technologies which can be used to deliver learning (or learning materials) in an electronic format, most likely via the internet (Gerhard & Mayr, 2002 ).Within the same line of defining e-learning as the delivery of the content through the technical channels, Paulsen more generally describes online learning as "the use of a computer network to present or distribute some educational content" (Paulsen, 2002).Psaromiligkos and Retalis consider e-learning systems as those which utilize the internet as a delivery medium for static learning resources, such as instructional files, or as an interface into interactive content (Psaromiligkos & Retalis, 2003).
The previous definitions look at e-learning in general.In more detail, e-learning can be seen in the form of courses or in the form of modules [separate parts of course's objects] and smaller learning materials.In addition, e-learning can include synchronous or asynchronous interaction.
Considering that there are two main types of e-learning: asynchronous and synchronous, depending on the interaction between learner and teacher, we will now discuss these in more detail.Synchronous e-learning environments require tutors and learners or the online classmates to be online at the same time, where live interactions take place between them.However, the focus of our research will be on the case where students are logging into and using the system independently of other students and staff members.This fits firmly into the general definition of the asynchronous e-learning environment.In this context, Doherty describes an Asynchronous Learning Network [ALN] as a variety of e-learning systems which distribute learning materials and concepts in one direction at a time (Doherty, 1998).Moreover, Spencer and Hiltz express ALN as a place where learners can interact with learning materials, tutors and other learner/s through the internet at different times and from different places (Spencer & Hiltz, 2001).
The position adopted in this research is that e-learning covers the technology used to distribute the learning materials, the quality of these materials, and the interaction with learners.We adopt in the definition of e-learning used in this paper these dimensions as described by the European Commission in (Gerhard & Mayr, 2002 "the use of new multimedia technologies and the internet to improve the quality of learning by facilitating access to resources and services as well as remote exchange and collaborations"

The concept of quality in e-learning systems
The definition of e-learning adopted in this research represents three fundamental dimensions: technology, access and quality.However, the focus in our study will be on quality, which is considered a crucial issue for education in general, and for e-learning in particular.Currently there are two recognized challenges in e-learning: the demand for overall interoperability and the request for [high] quality.Moreover, quality cannot be expressed and set by a simple definition, since in itself quality is a very abstract notion.The specified context and the perspectives of users need to be taken into account when defining quality in e-learning.It is also essential to classify suitable criteria to address this quality (Stracke, 2006).
Although it is important to set standards for information quality, this is a difficult and complex issue because there is no formal definition of information quality, as quality is dependent on the criteria applied to it.Furthermore, it is dependent on the targets, the environment and from which viewpoint we look at the information quality, that is, from the provider or the consumer perspective.This section of the paper will discuss concepts of quality in e-learning generally.Despite efforts to reach a comprehensive, universal definition of quality in e-learning, there is still a fundamental ambiguity surrounding it and we will approach this further in the paper's conclusions.
One position is to consider quality as an evaluation of excellence, a stance that is primarily adopted by universities and education institutions.For example, in universities, quality teaching and learning are promoted as the top priority, giving less attention to criteria or measurements regarding teaching input into courses, the learning outcomes, and the interactivity with the system (Crisp, 2002).Another trend is to consider the improvement in quality, by moving beyond the set of conceptions in the direction of flexible processes of negotiation which needs a very high level of quality capability from those involved (Ehlers, Goertz, Hildebrandt, & Pawlowski, 2005).
Quality can be viewed and considered from different aspects.In this context the SunTrust Equitable report (Close, Humphreys, & Ruttenbur, 2000) illustrates what they perceive to be the value chain in e-learning in the form of a pyramid.Content is the most critical factor of e-learning as it forms the base of the value pyramid.In fact, to be able to use the internet as a tool to improve learning, the content should not distract learners, but increase their interest for learning.Learning tools and enablers are also important in the learning procedure.In reality, providers of learning platforms and knowledge management systems are key factors in the successful delivery of content.The providers need infrastructure to deliver learning content.Moreover, learning service providers [LSP] are the distribution channels for content providers.One of the challenges which face these knowledge hubs and LSPs is to ensure that the learners are receiving fresh content.Companies focused on educational e-tailing [electronic retailing] are completing the value pyramid of e-learning.
From their e-learning value pyramid it can be observed that content is the most critical component of learning through the internet.In a similar manner, we will find that the measurement of the quality of content delivered by e-learning is the most important criteria and the most influential in the overall level of learning quality.

Information quality frameworks
Although it is important to set standards for information quality, it is a difficult and complex issue particularly in the area of information systems because there is no formal definition of information quality, as quality is dependent on the criteria applied to it.Furthermore, it is dependent on the targets, the environment and from which viewpoint we look at the information quality, that is, from the provider or the consumer perspective.Moreover information quality is both a task-dependent and a subjective concept; Juran summarises these aspects of quality in his quality definition as "fitness for use" (Juran., 1974).However, it is common to define information quality on the internet by identifying the main dimensions of the quality.For that purpose information quality frameworks are widely used to identify the important quality dimensions as described by Porter (Porter, 1991).
During the last years, much work has been done [as will be discussed later in this section] to build quality frameworks for information quality dimensions.In the past, research in information quality frameworks focused on data quality, but due to the recent development of internet technologies, information systems today are providing users information, not only data.Therefore, research attention shifted to focus on information quality frameworks.However, still in some studies the term "information quality" is interchangeable with "data quality".Discussion on this issue is outside the scope of this paper, but we will return to it in the future work.This part of the paper focuses on the Wang & Strong's data quality framework and reviews quality models, which were published since.We also present our proposed framework, which will be a result of the expansion of the original model to support identifying the key dimensions for information quality in e-learning systems.

Wang and Strong data quality framework
Wang & Strong's data quality framework, one of the most comprehensive, popular, remarkable and cited data quality framework, was established by Richard Wang and Diana Strong in 1996 (Wang & Strong, 1996).Their framework was designed empirically by asking users to give their viewpoint about the relevance of the information quality dimensions to capture the most important aspects of data quality to the data consumer.Lately, several quality management projects in business and government have successfully used this framework.Their hierarchical conceptual framework of data quality is shown in Figure 1 1 .
In their framework, Wang and Strong classified quality dimensions into four groups (Wang & Strong, 1996):  Intrinsic data quality: refers to the quality dimensions originated from the data in its own.This aspect of quality is independent of the user's perspective and context.
 Contextual data quality: focuses on the aspect of information quality within the context of the task at hand.In this group, the quality dimensions are subjective preferences of the user.Contrary to the first group, data quality dimensions cannot be assessed without considering the users viewpoint about their use of provided information.
 Representational data quality: is related to the representation of information within the systems.
 Accessibility data quality: refers to the quality aspects concerned into accessing distributed information.

Figure 1. Wang & Strong's data quality framework
Although their quality model will provide a good basis for our research to measure information quality in e-learning systems along the dimensions of this framework, it should be extended to include any undiscovered quality dimensions that occurred in the lately published research in the area of the quality in information systems.

Information quality in recent years
After Wang & Strong's data quality framework, diverse research efforts were spent in order to identify information quality dimensions in different contexts as shown in Table 1.
We extended Wang & Strong's data quality framework by examining seventeen frameworks within the recently published literature.In general, we found that there are nineteen quality dimensions permanently used in most of the frameworks.Fifteen of them are already used in Wang & Strong framework.Table 1 summarises the occurrence of these dimensions within the examined frameworks.Table 2 gives the frequency of the appearances for every dimension along the examined frameworks.
These dimensions are grouped into four categories as defined within the Wang & Strong's framework.The nineteen initial quality dimensions, which were identified in the examined frameworks, will be used as an extended framework and therefore as a fundamental base to discover the important quality dimensions from the users' perspective in the context of e-learning systems.

Table 1. Comparison between the emergences of quality dimensions in different information quality framework (part 1)
Table 1.Comparison between the emergences of quality dimensions in different information quality framework (Part 2)

The proposal for the extended framework
We propose to update Wang and Strong's data quality framework initially comprising another four quality dimensions.Therefore, the extending quality framework consists of four quality factors and nineteen quality dimensions as shown in Figure 2.

The survey
Although quality frameworks help in the measurement procedure, defining the quality using a framework is not enough because as mentioned before information quality is dependent on the application context.For that reason the identified quality dimensions were arranged in a questionnaire format to determine the users' view of the relative importance of quality dimensions in an e-learning system.This questionnaire1 seeks to gather the views of end-users about the importance of information quality dimensions in e-learning systems.It also gives an indication about the importance and relevancy of these quality dimensions for the users, which will help in ranking these dimensions in order to develop an information quality framework for quality metrics to measure the quality of information provided by e-learning systems.
This investigation was a cross-section survey performed on a sample chosen from a population of persons involved in academic work and dealing with e-learning systems in a regular basis.Respondents were included both of learners and teachers.The questionnaire was distributed to the respondents via e-mail because of its reduced cost, decrease short transfer time and its convenience for respondents.Surveymethods.com, an online survey software application, was used to create the survey, deploy it via e-mail, and collect and analyze respondent data through its graphical based analysis module.The questionnaire was planned to take less than five minutes to complete.The questionnaire consisted of three parts: Part 1 gives a brief profile of the respondent.
Part 2 addresses the user's attitude and usage of the internet in general and elearning systems specifically.
Part 3 asks respondents to rank the nineteen quality dimensions in order of their importance.

Figure 2. The extended framework
We collected responses from 315 e-learning system users 1 , from 24 different countries, 46% of the respondents were from Saudi Arabia, 26% from United Kingdom, 12% from Romania and the rest of the respondents were from the 21 remaining countries.57% of the participants were females, and 43% were males.All the respondents in the sample were e-learning users from different learning institutes.Of the respondents that contributed, the majority [66%] use e-learning as learners, and 29% as teachers and authors of the learning materials while 5% use e-learning systems for other purposes such as librarians and technicians.In addition, participants are holding various qualifications, 40% were holding Bachelor's degree, 33% have Master's degree, and 20% have PhD while the remaining 7% hold those listed as others.

Data analysis
We analyse the collected data from the third part of the questionnaire using SPSS to identify the most important quality dimensions in the area of e-learning systems and to build the final quality framework.
First, we conducted a frequency analysis for each variable to check for major mistakes and missing values.The results for variables frequency analysis in each dimension show that the data is valid and ready to be analysed.Reliability is the level to which research results would be the same if the investigation was to be repeated with a different sample or at a later date.In this research, the most accepted test of inter-item consistency reliability is the Cronbach's coefficient alpha (L.J. Cronbach, 1951; L.J. Cronbach, 1971).Based on Sekaran reliabilities less than 0.6 are considered to be poor, those in the 0.7 range are acceptable, and those over 0.8 are good (Sekaran, 2000).The closer to 1.0 the better the reliability coefficient is.It is generally agreed that the minimum acceptable value of Cronbach's alpha is 0.70 (Pallant, 2005;Peter, 1979), but this could be reduced to 0.6 for exploratory research (Robinson, Shaver, & Wrightsman, 1991).The Cronbach's alpha values for the dimensions in each quality factor gave an acceptable reliability level with 0.712, 0.735, 0.781, and 0.625 for intrinsic, contextual, representational and accessibility information quality respectively.
Screening the data responding to Churchill's recommendation will increase the reliability levels (Churchill & Gilbert, 1979).So, the collected data was screened by discarding items that showed very small corrected item-total correlations [<0.40].Because of this test, we delete timeliness and value-added variables from contextual factor, and access security variable from accessibility factor, which leaves only 16 dimensions in the framework.As a result, the reliability coefficient increased to 0.712, 0.748, 0.781, 0.668 for intrinsic, contextual, representational and accessibility factors respectively.
The next stage was conducting a factor analysis procedure with varimax rotation to check the dimensionality of the construct.To choose the cut-off value, there is no fixed measure.It depends on the purpose of the study on hand.Haire recommended that item loadings >0.30 are considered significant, >0.40 are more important, and >0.50 are considered very significant (Hair, Tatham, Anderson , & Black, 1998).While the aim of this study is to recognize the most important and significant quality attributes, we decided to use a cut-off point of 0.50 for item loadings and eigenvalue of 1.
The determinant of the correlation matrix1 is 0.002, which is greater than the necessary value of 0.00001.As a result, we are confident that multicollinearity will not cause any problems for our data (Field, 2000).

Table 3. KMO and Bartlett's Test Table 4: Total Variance Explained
The Kaser-Meyer-Oklin [KMO] measure of sampling adequacy and Bartlett's test of sphericity are illustrated in Table 3.The KMO static is a value between 0 and 1.A value close to 1 indicates that patterns of correlation are fairly compact and as a result factor analysis should gives distinct and reliable factors (Field, 2000).Values between 0.5 and 0.7 are average, values between 0.7 and 0.8 are good, values between 0.8 and 0.9 are great and values above 0.9 are excellent (Hutcheson & Sofroniou, 1999).Moreover, the significant value for the Bartlet's test should be less than 0.05 (Field, 2000).In our data, the value is .879,which is in the range of being great and the Bartlett's test is highly significant.Therefore, we are confident that factor analysis is appropriate for our data.
In addition to examining the overall KMO statistic, it is essential to check the diagonal elements of the anti-image correlation matrix 1 [which illustrates the KMO value for individual variables]; as in the overall KMO value these values have to be greater than 0.50 for all variables (Field, 2000).For our data, the values are in the range between [0.828 … 0.934].
Concerning the sample size, Comrey and Lee stated that 300 is a good sample size for factor analysis, 100 is poor while 1000 is excellent (Comrey & Lee, 1992).Since the number of our sample exceeds 300 respondents we should be confident that the sample size is appropriate for this type of analysis.4. Thus, the most suitable way is to stick with three factors.

Figure 3. Scree plot
Table 5 shows the rotated component matrix, which is the matrix of the factor loadings for each variable onto each factor.Factors loading less than 0.5 have not been displayed because we asked for these lodgings to be suppressed.As a result, we discarded these suppressed variables, which are Consistency and Interpretability, which leave only 14 variables in total.
Analysis finding shows that there are three information quality factors in elearning systems not four, as proposed previously.We recognized that contextual and 1 See appendix II representational quality factors are measuring the same aspects from e-learning systems users' perspective.Therefore, we propose a new framework, with 14 dimensions of information quality in e-learning systems to measure three quality factors: Intrinsic, Contextual representation and Accessibility information quality.
We then calculated Cronbach's alpha values for the variables in each new factor, which gives a good reliability level with 0.842, 0.697, and 0.665, for Intrinsic, Contextual representation and Accessibility information quality respectively.The new proposed framework is shown in Figure 4.

Table 5. Rotated Component Matrix a
Linear regression then was used to predict the factor scores from the variables.correlation coefficient, and can be obtained by squaring the "part corr" provided by SPSS "β in the equation bellow".For example for Completeness in the first factor, that is 0.156 2 = 2.434%.These statistics will sum to less than 100%.To get them to sum to 100%, we divided each by the sum of all.So we can calculate the relative importance for each variable in the correlated factor we can use the following equation: where βi is the partial correlation for the variable in the corresponding factor.The same logic was conducted to define the relative importance for each factor in the overall quality.
The zero-order correlations in Appendix3 are the loadings.One could define the relative importance of a variable as the amount by which the explained variance in the factor is reduced if the variable is removed from the regression model.That statistic measure is the squared semi-partial.

Conclusion and future work
Based on original questionnaire data and factor analysis, we proposed a new quality framework to measure the quality of the content provided by e-learning systems.Linear regression was used to calculate the relative importance for each dimension inside the three quality factors, and the relative importance for each factor in the overall quality.This framework could be used to provide a comprehensive indication of information quality in the context of e-learning systems.It could be useful to e-learning systems designers, providers and users as it provides a comprehensive indication of the quality of information in such systems.
As mentioned before, the framework proposed in this paper is a part of a larger research project.The next stage will be the development of a set of quality metrics and an experiment to compute these metrics in chosen e-learning systems.The value calculated for each metric will then be compared with the results from a user satisfaction survey.The research also will focus on taking advantages of software agent technologies in order to automate data collection and evaluation processes.
Before extraction, SPSS has identified 16 factors within the data set.SPSS then extracts all factors with eigenvalues greater than 1, which leave us with three factors.From the scree plot shown in Fig 3, we can see that the point of inflexion on the curve on three factors which is in conformity with the results shown in Table

Figure 4 .
Figure 4.The new proposed framework

Figure 5 .
Figure 5.The revised framework

Table 4
lists the eigenvalues associated with each factor before extraction, after extraction and after rotation.