Learning analytics: A comparison of western, educated, industrialized, rich, and democratic (WEIRD) and non-WEIRD research

: We examined how Learning Analytics literature represents participants from diverse societies by comparing the studies published with samples from WEIRD (Western, Industrialized, Rich, Democratic) nations versus non-WEIRD nations. By analyzing the Learning Analytics studies published during 2015-2019 ( N = 360), we found that most of the studies were on WEIRD samples, with at least 58 percent of the total studies on WEIRD samples. Through keyword analysis, we found that the studies on WEIRD samples’ research topics focused on self-regulated learning and feedback received in learning environments. The studies on non-WEIRD samples focused on the collaborative and social nature of learning. Our investigation of the analysis tools used for the studies suggested the limitations of some software in analyzing languages in diverse countries. Our analysis of theoretical frameworks revealed that most studies on both WEIRD and non-WEIRD samples did not identify a theoretical framework. The studies on WEIRD and non-WEIRD samples convey the similarities of Learning Analytics and Educational Data Mining. We conclude by discussing the importance of integrating multifaceted elements of the participant samples, including cultural values, societal values, and geographic areas, and present recommendations on ways to promote inclusion and diversity in Learning Analytics research.


Introduction
With the vast development of technology over the past few decades, big data has become readily available in educational settings, which can be used to understand the learning processes of students at a fine-grained level (Ang et al., 2020;Baek & Doleck, 2020;Lemay & Doleck, 2022).Further, innovative technology platforms have been developed and deployed continually in various educational settings (Choudhury & Pattnaik, 2020).Learning Analytics (LA) is a field that emerged in response to these new developments in technology for education (Siemens & Baker, 2012).A widely used definition for LA is: "the measurement, collection, analysis and reporting of data about learners and their contexts, for the purpose of understanding and optimizing learning and the environments in which it occurs" (Siemens & Baker, 2012).LA can optimize the learning processes and performances by providing individualized and personalized learning environments for students using techniques such as analyzing learners' progress in real time and providing timely feedback (Charitopoulos et al., 2020;Lemay et al., 2021).One of the challenges of LA has been generalizability, inclusion, and diversity (Baker, 2019;Gasevic et al., 2016;Mathrani et al., 2021).For example, building a prediction model from a set of data (Doleck et al., 2019) from a particular sample of students is not generalizable to another group of students (Baker, 2019;Mathrani et al., 2021).Models and platforms designed for a particular group of students can undermine the learning opportunities for the groups that were excluded in the process (Baker, 2019;Bayer et al., 2021).Therefore, it is timely and crucial to examine LA research's representation of samples from different groups, particularly from WEIRD and non-WEIRD societies.The acronym WEIRD stands for Western, Educated, Industrialized, Rich, or Democratic, as defined by Henrich et al. (2010).A recent criticism of the behavioral and social sciences literature is related to the fact that most of the samples in the literature were from WEIRD countries, although WEIRD countries make up a small portion of the entire world (Dogruyol et al., 2019).Similarly, previous studies have found that technology-focused literature over-represents WEIRD groups.Blanchard (2012) analyzed papers published in Intelligent Tutoring Systems and Artificial Intelligence in Education Conferences during 2002-2011, which revealed a dominance of WEIRD samples, with samples from the United States constituting 61% of total studies.Similarly, Linxen et al. (2021) found that 73% of papers published at the ACM CHI Conference on Human Factors in Computing Systems, a premier venue for Human-Computer Interaction, were on Western samples.
A field that over-represents WEIRD samples inevitability yields results that may undermine diversity, inclusion, and equity, as previous studies have shown that study results for WEIRD and non-WEIRD samples differ.Henrich et al. (2010) found that databases on behavioral science domains, such as reasoning styles and visual perception showed variability between WEIRD samples and non-WEIRD samples.Laajaj et al. (2019) documented that the Big Five personality measure that was validated with WEIRD samples failed to work for non-WEIRD samples, showing how a measure that is validated for specific groups of people may not generalize to other groups.The aforementioned findings exhibit evidence of variation across human populations in diverse research contexts and corroborate that inclusive research must integrate members from diverse societies (Ekuni et al., 2020).
Overrepresenting WEIRD samples in a field disregards important perspectives, attitudes, and needs of stakeholders from non-WEIRD countries, which poses a significant challenge for emerging fields like LA that constantly introduce and implement new technologies in educational environments.To address this issue, it is crucial to engage in more discussions that incorporate the viewpoints and experiences of stakeholders from WEIRD countries.For example, examining faculty members' attitudes towards online learning in non-WEIRD countries (e.g., Ogbodoakum et al., 2022) or exploring teachers' perceptions of using virtual reality (e.g., Çoban et al., 2022) can provide valuable insights on designing and implementing inclusive LA-integrated systems.

Purpose of the study
As LA continues to evolve as a research field with the aim of providing individualized and personalized learning environments to optimize learning for all students, it is crucial to examine the recent trend of LA research on its inclusiveness of diverse societies and the outcomes of research with participants from different societies.To our knowledge, there has not been a study that compared LA studies on WEIRD samples versus non-WEIRD samples.Thus, we aim to fill this gap by adopting the framework of Henrich et al. (2010) in examining the LA studies published during 2015-2019 to analyse the representation of WEIRD samples and non-WEIRD samples in LA literature as well as to compare the overall trend of the studies on WEIRD versus non-WEIRD samples.We chose the period of 2015-2019 as LA literature grew notably during this period, such as the launching of the Journal of Learning Analytics in 2014 and the notable increase in the volume of publications over the five years (Baek & Doleck, 2021).
We aim to answer the following research questions: RQ1: Are there differences in the volume of the publications between studies focused on samples from WEIRD versus non-WEIRD countries published during 2015-2019?RQ2: Are there differences between the common keywords of studies focused on samples from WEIRD versus non-WEIRD countries published during 2015-2019?RQ3: Are there differences between the data analysis tools of studies focused on samples from WEIRD versus non-WEIRD countries published during 2015-2019?RQ4: Are there differences between the theories used in studies focused on samples from WEIRD versus non-WEIRD countries published during 2015-2019?RQ5: Are there differences between the definitions of Learning Analytics used in studies focused on samples from WEIRD versus non-WEIRD countries published during 2015-2019?By examining the volume of the publications between studies focused on samples from WEIRD versus non-WEIRD countries, we aim to investigate the extent of LA research's representation of diverse populations around the globe.Comparing the differences between the common keywords of studies, theories, and definitions of Learning Analytics can help surface research trends and prevalent topics in WEIRD versus non-WEIRD LA research.If there is a difference in prevalent topics, we need to examine whether students from non-WEIRD countries are excluded from discussions on particular topics.Comparison of tools, theories, and definitions of Learning Analytics would shed light on whether the methods and framework of LA research are generalizable to represent students with diverse characteristics in different settings and if there is a need to develop methods to best study geographically and culturally diverse samples (Linxen et al., 2021).

Search and collection of articles
The search and collection of articles took place in November 2019.First, we used the Web of Science database (Web of Science Core Collection) to collect the Learning Analytics articles by setting the filter to 2015-2019.Second, we used the search terms "Learning Analytics" (TOPIC: ("Learning Analytics") AND DOCUMENT TYPES: (Article)).This resulted in the initial hit of 850 sources.Third, we screened the articles using our inclusion and exclusion criteria (see Table 1), resulting in 492 articles.• The primary purpose is not exploring educational context (e.g., developing machines not related to learning)

Coding scheme for countries
Next, we coded for the countries of the sample from each study.We read each paper to find the countries where the study took place or where the datasets were collected.The information on the location of the study or the origin of the datasets was usually described in the methods section.If we could not find the information in the methods section, we conducted a closer read of the papers to find the information in other sections of the papers.If we could not find the information after a closer read, we contacted the authors of the papers for the information.We emailed 27 authors and 14 of the authors responded.

List of WEIRD and non-WEIRD countries
To compile a list of WEIRD and non-WEIRD countries, we modified the chart of the WEIRD nations developed by Many labs 2: Investigating variation in replicability across sample and setting by Klein et al. (2018) that was publicly shared on Open Science Framework.For each WEIRD category (Western, Industrialized, Educated, Rich, Democratic), we used the available sources to assign a score for each country.Each source is described below.
Western: The list of Western countries was compiled by the World Population Review (2024), which lists 68 Western countries.We used this list to assign 1 for the Western countries and 0 for the non-western countries.Educated: We used the Education Index compiled and shared by the United Nations Development Programme (n.d.).We chose the education index of the year 2013.The most recent MVA value for the industrialized category that we were able to locate was from 2013.Thus, to ensure that the years of the categories are consistent, we chose the values from 2013 onwards on the Education Index List.
Rich: We used the "Economies by per capita GNI in 2012" table from the country classification document provided by the United Nations ( 2014).The table has four categories with the corresponding countries listed in each category: high income, upper middle income, lower middle income, and low income.We assigned 1 for the countries listed in the high-income category and 0 for the rest.For this list of countries' economies by per capita GNI, the year 2012 was the most recent list we were able to locate.
Democratic: We used the Democracy Index from Wikipedia, which was compiled by the data from the Economist Intelligence Unit (EIU) (Wikipedia, 2024).The Democracy Index list consists of each country's democracy score in each year.The values of the democracy index range from zero to ten and are scored based on pluralism, civil liberties, and political culture.

Collection of WEIRD and non-WEIRD countries
Using the LA articles in our collection, we coded the country of the sample of each study in our collection as either WEIRD or non-WEIRD countries using the list of WEIRD nations (see Table 2).The overall mean WEIRD score of the list was 0.56 after we updated the values of each WEIRD category for all countries.We coded the countries with scores below the overall mean weird score as "Non-WEIRD," and the countries with values above the overall mean weird score as "WEIRD."A total of 132 studies were excluded for one of the following reasons.First, studies with datasets collected from international samples around the globe were excluded.Second, studies that did not include human subjects were excluded.Third, studies that did not list the country of the samples and the authors did not respond to requests for the country names were excluded.Also, there was just one study that identified the sample from both WEIRD and non-WEIRD countries and this study was excluded for consistency.The final collection had 360 articles, with 91 non-WEIRD articles (Studies with non-WEIRD samples) and 269 WEIRD articles (Studies with WEIRD samples).Fig. 1 illustrates the entire screening process of the articles.

Data analysis
To answer our research questions that examine the difference between WEIRD and non-WEIRD studies, we coded and analysed the keywords, tools, theories, and definitions of each article of the collection.We first examined the authors' keywords listed in the articles, as analysing keywords implies the research topics that the studies focus on.Then, we analysed the tools that each study used to analyse their results as the type of tools used can reveal the methodologies the studies used.We investigated the theories identified in the studies to reveal the theoretical frameworks that guided the studies.Last, we explored how the studies define LA.In examining the definitions, we focused on how the studies distinguished the definition of Learning Analytics from a closely related field, Educational Data Mining (EDM).Examining the definitions will reveal how LA research has been evolving.We conducted a separate analysis for the WEIRD articles set and the non-WEIRD articles set for each research question.

Keywords
To examine whether there exists a difference in research topics between WEIRD and non-WEIRD studies, we collected the author's keywords identified in each study which we found below the abstract.Then, we separated the list of keywords into two sets: WEIRD and non-WEIRD.Thirty-seven studies in the entire collection of articles did not list keywords and were excluded from this analysis.We conducted a frequency analysis in each set using the tidytext package in R Studio.Specifically, we examined the frequently occurring single keywords and binary keywords.

Tools
To answer our research question about the difference between WEIRD and non-WEIRD studies on the tools used for analysis (e.g., name of software used to analyse data), we reviewed each article to find the information on the tools.This information on tools was usually described in the methods section of the studies.When authors listed multiple tools used for the studies, we counted them separately.For the studies that did not list the tools used, we coded them as "tools not listed".

Theories
To answer our research question on theories used by the studies, we followed the coding scheme Baek and Doleck (2021) created by adopting the coding scheme of Hew et al. (2019).We followed three steps in our coding and analysis of theories.Using the coding scheme in Table 3, we coded each article as "theory listed" or "theory not listed."For each article that identified a theory, we listed the name of the theory.Then, we conducted a frequency analysis of the identified theories.

Definitions
To analyse how the studies defined LA, we first extracted the definitions for LA and EDM from the articles.We did not include the definitions defined under other branches of LA, such as Predictive Analysis and confined the collection of definitions to the general categories of LA.We excluded the articles that did not provide a definition of LA.After collecting all the LA definitions, the three cases emerged in the way the articles defined LA: authors define LA only, authors discuss the similarities and differences between LA and EDM, and authors define LA and EDM collectively.Using these three cases, we built a coding scheme and followed this coding scheme to code each article to one of the three cases (see Table 4).Used the theory for data collection or analysis procedure, used the theory for framing the structure of the study, used the theory to discuss the research outcomes, used the theory for supporting authors' arguments Theory not listed Listed theories, theoretical framework, theoretical model, conceptual framework, or theoretical perspective without explaining how they used the theories in relation to their study Listed some theories in the literature review section without mentioning how these theories are related to or used in the study No mention of theories at all N/A • Authors combine the two fields, LA and EDM, as one word (e.g., "LA/EDM is…")

Number of WEIRD and non-WEIRD studies
There are 269 studies focused on the WEIRD samples and 91 on the non-WEIRD samples.Fig. 2 shows the publication distribution of the 2015-2019 period of WEIRD versus non-WEIRD.Fig. 2 shows that there is a clear difference between the number of publications of WEIRD studies and non-WEIRD studies.Every year from 2015 to 2019, the number of publications for WEIRD studies is more than twofold the number of publications for non-WEIRD studies.For WEIRD studies, Computers in Human Behavior was the most frequently published venue (N = 22) whereas only a few studies were published in Computers in Human Behavior for non-WEIRD studies (N = 3).Interactive Learning Environments were the most frequently published venue (N = 11) for non-WEIRD studies.
The aims and scope of Computers in Human Behavior emphasize the use of computers from a psychological perspective, whereas the aims and scope of Interactive Learning Environments relate to publishing articles on learning environments from a broader perspective.

Fig. 2. Number of publications on WEIRD and non-WEIRD Samples
Fig. 3 and Fig. 4 show the most frequently studied WEIRD and non-WEIRD country samples, respectively.Fig. 3 shows that the samples in the United States are most frequently studied, followed by Spain.Fig. 4 shows that samples from China and Taiwan are the most frequently studied.

Keywords
Our analyses of the keywords show that there are differences, as well as similarities between the studies, focused on WEIRD samples versus non-WEIRD samples.As expected, keywords pertaining to LA are the most frequently occurring keywords for both WEIRD and non-WEIRD studies (see Fig. 5).Specifically, "learning," "analytics," and "data" are the top three occurring keywords for both WEIRD and non-WEIRD studies.The words "social" and "collaborative" are in the top 30 frequently occurring keywords for both WEIRD and non-WEIRD studies.For non-WEIRD studies, "collaborative" and "social" are in the top 10 most frequently occurring words."English" is one of the top occurring keywords for the non-WEIRD studies, whereas it is not for the WEIRD studies.Also, "feedback" is a common keyword in the WEIRD studies but not in the non-WEIRD studies.
Fig. 5. Top 30 Frequently occurring keywords for WEIRD and non-WEIRD Fig. 6 shows the most frequently occurring bigrams for the WEIRD studies and the non-WEIRD studies.The bigrams for the non-WEIRD and WEIRD studies include "big data" and "social network."As shown in the single keywords analyses, the binary keywords of the non-WEIRD studies and the WEIRD studies have "social network" and "collaborative learning" at the top of the lists."Higher education" is a frequently occurring word for the WEIRD studies but not for the non-WEIRD studies."Self-management" is one of the top single keywords for the WEIRD studies and similarly, "self-regulated" is one of the top binary words for the WEIRD studies.

Theories
The WEIRD studies and the non-WEIRD studies fall into one of two categories (as mentioned in Table 3 before): (1) theories used to frame the study are explicitly stated (e.g., authors name the theory and clearly describe how they used the theory for their study); (2) theories are stated without making a meaningful connection to the study (e.g., authors name multiple theories without making a meaningful connection to their study); no theories are mentioned in the study at all.About 63 percent of the non-WEIRD studies did not identify theories.Similarly, about 60 percent of the WEIRD studies did not identify theories.The theories that were most frequently used by non-WEIRD studies include self-regulated learning theories and cognitive theories.Self-regulated learning theories, social theories, constructive theories, and cognitive theories were used by the WEIRD studies.

Tools
For both WEIRD and non-WEIRD studies, most did not list the tools used for the analysis.Some of the studies identified multiple tools used for the analysis, and these tools were counted separately.About 55 percent of the non-WEIRD studies did not list the tools and about 65 percent of the WEIRD studies did not list the tools.The tools identified by the WEIRD studies consist of 33 different tools, and the non-WEIRD studies consist of nine different tools.Fig. 7 shows the most frequently listed tools for WEIRD and non-WEIRD studies.SPSS and R/R studio are the most frequently listed tools for WEIRD and non-WEIRD studies.Especially for the non-WEIRD studies, SPSS has a notably higher frequency than all the other tools.The third and fourth frequently listed tools for the WEIRD and the non-WEIRD studies differ.NVivo and SAS are the next frequently listed tools for the WEIRD studies, whereas WEKA and Gephi were for the non-WEIRD studies.

Definitions
In defining LA, both WEIRD studies and non-WEIRD studies mostly defined LA only (Fig. 8 and Fig. 9).Some WEIRD and non-WEIRD studies defined LA as well as EDM by distinguishing them.For example, in a study that focused on a WEIRD sample, Gelan et al. (2018), defined LA as the measurement of data about learners in their context to understand learning and EDM as a field driven by data emphasizing learning and teaching.Gelan et al. (2018) also emphasized that LA needs to be distinguished from EDM.Similarly, a study focused on a non-WEIRD sample, Dragulescu et al. (2015), addressed the key distinction between EDM and LA.Dragulescu et al. (2015) discussed that EDM focuses on the automatic process and LA focuses on human judgment.Some WEIRD and non-WEIRD studies defined LA and EDM collectively.For example, in their study that focused on a WEIRD sample, Polyzou and Karypis (2019) addressed the similarities between the two fields by stating that both EDM and LA have been developed to provide tools for supporting the learning process.

Discussions
Our findings show that most of the Learning Analytics studies published during 2015-2019 are drawn from WEIRD samples.According to our results, 268 studies were on WEIRD samples, which is at least 58 percent of the total number of Learning Analytics studies (N = 492).The studies on WEIRD samples constituting most of the literature align with the previous studies that examined the proportion of WEIRD representation in computer technology-related literature.Blanchard's (2012) analysis of papers published in ITS and AIED conferences revealed the dominance of WEIRD samples.Similarly, Linxen et al. (2021) examined the publications of a premier venue that publishes Human-Computer Interaction research and found that most participant samples were from WEIRD countries.
The LA community wears a similar imbalance in representing WEIRD samples and non-WEIRD samples like the AIED and HCI communities.This imbalance implies that most of the findings from LA research are derived from WEIRD samples, consequentially underrepresenting the experiences and needs of learners in non-WEIRD countries.However, as Fig. 2 shows earlier, the volume of publications on non-WEIRD countries increased over the five years, and the gap between WEIRD and non-WEIRD studies decreased over the years.The increase in publications on non-WEIRD samples suggests that LA research is starting to represent more diverse populations.
We also found that some countries are overstudied within the WEIRD country studies.The United States and Spain together make up 45% of the WEIRD studies, which accounts for almost half of the WEIRD studies.Further, as discussed in the previous section, only one study included both WEIRD and non-WEIRD samples.Other studies that included samples from multiple countries had samples from other non-WEIRD or other WEIRD countries.Thus, this suggests the limited international collaboration between WEIRD and non-WEIRD countries.This finding aligns with the analyses of publications in related fields, Artificial Intelligence and Educational Data Mining, which showed limited international collaboration and WEIRD countries actively collaborating (Baek & Doleck, 2020;Baek & Doleck, 2022).One way to increase the representation of non-WEIRD samples is to increase international collaboration between authors in WEIRD and non-WEIRD countries as authors tend to locally recruit samples which in turn could lead to increased inclusion of non-WEIRD samples (Linxen et al., 2021).
Our analysis of the single keywords for the studies on WEIRD and non-WEIRD samples exhibits differences as well as similarities between the two sets of studies.Both non-WEIRD and WEIRD studies exhibited a focus on collaborative learning environments."English" is one of the top-occurring keywords only for non-WEIRD studies.This suggests the non-WEIRD studies' active engagement in developing systems to teach English.For example, Lin (2019) developed an online peer assessment to teach English to college students in Taiwan using a flipped classroom method.This trend reflects that LA has been used for studying English in countries that do not predominantly speak English.The word "feedback" only frequently occurs in WEIRD studies, which suggests high involvement of computer-based environments for these studies as "feedback" often pertains to feedback received from computer-based environments.For example, Cutumisu et al. (2018) examined a group of Canadian college students' eye movements to explore their cognitive processes when processing feedback.
The keywords of the WEIRD studies exhibit the prevalence of topics on selfregulated learning.Specifically, "regulated," "self," and "management" are frequently used single keywords for the WEIRD studies.Also, for binary keywords, "self-regulated" is one of the frequently used words.This trend of the WEIRD studies' focus on selfregulation may be tied to our previous discussion on the prevalence of the topic feedback, which collectively exhibits a trend of using feedback generated in computerized environments for learners' self-regulation.For example, Van Horn et al. (2018) investigated the relationship between U.S. university students' self-regulated learning and their access to feedback via a dashboard.On the other hand, the non-WEIRD studies focus on the collaborative and social nature of learning in computerized environments.The top binary and single keywords for the non-WEIRD studies contain social learning-related words: "network," "social," "engagement," "social network," and "collaborative learning."For example, Cheng et al. (2020) investigated the effectiveness of using a group leadership promotion approach in collaborative learning tasks to develop Taiwanese university students' creativity, problem-solving, and critical thinking skills.
Although there has been a lack of identification of tools used for analyses for both WEIRD and non-WEIRD studies, the identified tools reveal similarities and differences.The statistical software SPSS was the most frequently listed data analysis tool for WEIRD and non-WEIRD studies, which implies the common focus on quantitative research.R/R Studio is used for various computational methods to investigate data, including statistical analyses and machine learning while incorporating graphics.Again, the frequent use of R/R Studio implies the studies' focus on computational methods to analyse big data.NVivo, a software that analyses qualitative data, was a tool that was frequently listed by the WEIRD studies followed by another statistical analysis software, SAS.For example, Knight et al. (2018) used NVivo to identify thematic patterns of student feedback about the efficiency of a natural language processing tool that provides feedback on students' writing.According to our analysis, none of the non-WEIRD studies used NVivo.This contrasts with how the WEIRD studies frequently used NVivo.The difference may be due to how NVivo works for a limited set of languages: Chinese, English, French, German, Japanese, Portuguese, and Spanish.The languages in the list are spoken by many of the WEIRD countries, and the set is very limited.Thus, many of the studies with non-WEIRD samples could not use the software.The software that excludes non-WEIRD languages corroborates the argument on how English language dominance in research communities can be a contributing factor to WEIRD bias in research (Blanchard, 2012).
The prevalent use of Gephi in non-WEIRD studies implies a focus on collaboration and social aspects of learning.For example, Saqr and Alamo (2019) used Gephi to explore Saudi Arabian University students' social interactions during online Problem Based Learning through Social Network Analysis using Gephi.The prevalence of social and collaborative learning-related topics may be related to some of the non-WEIRD society's values on teamwork and socially orientated motivations.For example, a previous study on the WEIRD population by Datu and Bernardo (2020) acknowledged how achievement motivations are reported to be socially oriented in the society of the Philippines (non-WEIRD context).Datu and Bernardo (2020) found a significant association between high school students in the Philippines' interpersonal strengths (e.g., teamwork) and academic engagement, unlike the findings involving students from WEIRD settings that showed an inconsistent association.Our findings of the non-WEIRD studies also exhibit the importance of social values.
Both the tool and keyword analysis indicate a focus on social-related learning of WEIRD studies.Our findings emphasize the importance of considering the cultural and societal values of learners when implementing and developing learning analytics systems.Again, LA research predominantly representing learners in WEIRD countries is concerning as LA systems designed and tested in WEIRD settings would not best meet the needs of learners in non-WEIRD settings.
In incorporating the findings from WEIRD samples apropos learning processes and outcomes while integrating theories in computerized learning environments, stakeholders need to be cautious as the findings might differ in other socio-cultural contexts (Blanchard, 2012).Thus, studies must identify theoretical frameworks that guide the studies.According to our examination of how theories are identified, most of the studies on non-WEIRD and WEIRD samples do not identify theories or connect them to their studies.A lack of engagement with theory for education technology-related research is problematic as research on learning without a theoretical framework is constrained in the ability to connect the findings to practical pedagogical practice (Baek & Doleck, 2021;Pardo et al., 2016;Rabin et al., 2019;Reich, 2015).Especially for research that is largely represented by the WEIRD samples, the generalizability to different populations would be further limited when lacking theoretical guidance.
LA, a relatively nascent field, has been rapidly evolving over the years.Therefore, investigating how the literature defines the field can shed light on current and future research directions.About one-fourth of the WEIRD studies, as well as the non-WEIRD studies, defined EDM along with the LA either by distinguishing the two fields or addressing them collectively.This trend of defining LA and EDM shows the closeness of the two fields in all research settings and implies the overlap of the methods, techniques, tools, and research communities of both fields across different countries.
One valuable lesson we learned through the pandemic is the accessibility and efficiency of online collaboration.LA researchers can conduct studies with samples in other countries using online platforms to recruit and conduct studies, which would help include more diverse populations (Linxen et al., 2021).However, increasing diversity and mitigating bias in research are more complex processes than merely increasing the volume of studies on non-WEIRD populations (Kanazawa, 2020).The authors should describe the samples of their studies in more detail, such as whether the sample was from rural or urban settings (Ghai, 2021;Linxen, 2021).More detailed information about the sample will help with discussing multifaceted issues such as culturally relevant challenges (Blanchard, 2012).As new learning analytics-integrated systems are developed, researchers should evaluate the systems by identifying barriers that developing countries may experience in using the systems, continually assessing the systems, and accommodating the local culture in adopting the systems (Nye, 2015).
This study has several limitations.First, our study included only those studies that were published in English and subsequently could have excluded studies published in non-English that could have added valuable insights on the landscape of the learning analytics research for both WEIRD and non-WEIRD samples.Thus, we acknowledge that our collection of articles as well as the analysis results are not generalizable to all learning analytics publications in WEIRD and non-WEIRD countries.Also, some studies were excluded because we were not able to retrieve the information on the location of the study.Our study is limited to the LA literature published during 2015-2019.An examination of the literature during another period may yield different findings.Importantly, we acknowledge the limitations of the WEIRD and non-WEIRD framework such that it simply dichotomizes the world into two different groups.The WEIRD framework overlooks the heterogeneity of one country's population and how people in one country differ based on many different elements including social class, religious values, and cultural traits (Ghai, 2021;Linxen et al., 2021).Promoting diversity and inclusion in LA research is a far more complex process than addressing the overrepresentation of studies on WEIRD samples, However, we believe that our study is an initial step towards promoting diversity and inclusion in LA research since an important way to identify the bias and mitigate this issue in research is to make the community aware of it (Blanchard, 2012).
Industrialized: We used the Industrial Development Report 2016 which was published by the United Nations Industrial Development Organization (2016).We utilized Annex B2 in the document, which shows indicators of competitive industrial performance by the economy.We used MVA (Manufacturing Value Added) per capita for the year 2013.MVA per capita is the basic indicator of a country's level of industrialization adjusted for the size of the economy.The MVA values are mostly in the hundreds and thousands.For the consistency of the decimal values with the values in other categories of WEIRD, we converted each value by multiplying by .

Fig. 1 .
Fig. 1.Overall screening procedure for learning analytics articles

Fig. 7 .
Fig. 7. Top tools listed for WEIRD studies and non-WEIRD studies

Table 1
Inclusion and exclusion criteria for collecting learning analytics literature

Table 4
Coding scheme for definitions