Knowledge convergence among pre-service mathematics teachers through online reciprocal peer feedback

This research focused on pre-service mathematics teachers’ sharing of knowledge through reciprocal peer feedback. In this study, pre-service teachers were divided into groups of five and engaged in an online reciprocal peer feedback activity. Specifically, after creating an individual concept map indicating high school students’ possible solutions to an algebra problem, preservice teachers shared their individual maps with team members and engaged in online discussion, commenting on the concept maps of other group members and responding to peers’ feedback. Similarities in team members’ knowledge representations before and after this peer feedback activity were compared in order to analyze their knowledge convergence. It was found that a team member’s knowledge was more likely to match that of other team members after the online reciprocal peer feedback activity. Qualitative analysis was also conducted in order to explore the possible influence of a team’s interaction process on members’ knowledge convergence. It was also found that, after engaging in this peer feedback process, pre-service teachers demonstrated greater improvement in their convergence of concepts relating to problemsolving strategies than in the concepts representing problem context and domains.


Introduction
Reciprocal peer feedback, also named reciprocal peer review or reciprocal peer critique, indicates a communication process (Liu & Carless, 2006), during which learners comment on their peer's learning product or performance by identifying strengths and areas for improvement (Cho & Cho, 2011); meanwhile, students also receive feedback on their own product. Compared with peer assessment that involves grading of peer's performance, which some students feel uncomfortable about (Liu & Carless, 2006), reciprocal peer feedback is perceived as a less-threatening process and has been found to W. Chen (2017) benefit students' learning (Boase-Jelinek, Parker, & Herrington, 2013;Gielen, Peeters, Dochy, Onghena, & Struyven, 2010). The integration of peer feedback into instructional practices also enhances the frequency and timeliness of feedback provision without overwhelming instructors (Gielen, Peeters et al., 2010). An increasing number of studies have been conducted to examine the educational implications of peer feedback. However, most of them were performed within traditional classroom settings, and relatively fewer studies have been conducted in online environments (Ching & Hsu, 2013;Ertmer et al., 2007). In this project, pre-service mathematics teachers engaged in online discussion exchanging feedback about their team members' concept maps, and the outcome of their participation in this online peer feedback activity was examined.
Regarding assessment of the outcomes of peer feedback, an increasing number of studies have moved towards inspecting the deeper influence of this instructional activity, which is consistent with the trend in educational program evaluation (Kirkpatrick & Kirkpatrick, 2015). Specifically, researchers have gone beyond measuring learners' attitudes towards or perception of peer feedback per se, and have started investigating students' acquired knowledge and skills. However, as can be found in the general field of learning psychology (Fischer & Mandl, 2005), even though peer feedback constitutes one form of collaborative learning (Gielen, Peeters et al., 2010), most investigators have mainly examined individual students' learning achievement, and there has been a significant lack of studies analyzing team members' collective accomplishment. A successful collaborative learning process should lead to not only individual but also collective success, as learners construct knowledge together and integrate this shared understanding into individual mindsets. Thus, this study focused on pre-service teachers' knowledge convergence, a measure of their collective accomplishment. Knowledge convergence assesses the similarity in group members' knowledge representations after they have engaged in a collaborative learning activity (Jeong & Chi, 2007;Weinberger, Stegmann, & Fischer, 2007). Although studies have shown that peers are able to learn from providing and receiving feedback (as reviewed in 2.3), so far no research has been conducted to compare participants' learning outcomes and to assess their convergence of understanding through peer feedback. This project studied whether team members' knowledge became more similar after they had engaged in online peer feedback. Additional explorations of participants' interaction processes and their sharing of different types of knowledge were also conducted to supplement the understanding.

Peer feedback on concept maps
Feedback plays an essential role in enhancing students' learning achievements and motivations (Shute, 2008). Peer feedback is usually provided formatively; that is, rather than intending to grade the assessees (Liu & Carless, 2006), the main goal is to improve the recipient's knowledge or skills (Shute, 2008). Studies have shown that peer feedback promotes the learning performances of both assessors and assessees (Cho & Cho, 2011;Cho, Chung, King, & Schunn, 2008;Li, Liu, & Steckelberg, 2010;Liu, Lin, Chiu, & Yuan, 2001;Lu & Law, 2012;Xiao & Lucking, 2008). However, although extensive literature exists on the applications of reciprocal peer feedback in instructional tasks, including writing assignments and clinical simulations, few empirical studies have investigated the outcomes of asking participants to comment on their peers' concept maps. A concept map, or a semantic network, reflects its mapper's organization of knowledge about a specific topic and includes these two main elements (Cañas et al., 2003):


Nodes, each representing a concept; and  Labeled links, each connecting two concepts and describing the relationship between them. Each pair of concepts and their labeled link presents a proposition or a statement.
In this study, pre-service teachers engaged in concept mapping and in commenting on their team members' maps. CmapTools (cmap.ihmc.us) software, developed by the Florida Institute for Human & Machine Cognition, was adopted to facilitate the research participants' creation, modification, sharing, and commenting of their concept maps. This software has been widely used internationally to support students' concept mapping (IHMC, 2014).

Knowledge convergence
During collaborative learning, one challenge, for both the researchers and the practitioners, is achieving an understanding of how learners who began with different mindsets could reach joint understanding and think more alike (Roschelle, 1992). Studies show that groups' achievement of knowledge convergence is significantly associated with their learning outcomes (Fischer & Mandl, 2005;Jeong & Chi, 2007;Zheng, Chen, Huang, & Yang, 2014). Knowledge convergence, therefore, plays an important role in the success of collaborative learning and knowledge construction. However, the investigation of team members' achievement of knowledge convergence is currently still an emerging area for research. Additionally, existing explorations of knowledge convergence have mainly been conducted with synchronous collaborative activities, and little has been done to inspect team members' knowledge convergence through their participation in asynchronous collaborative activities, such as online reciprocal peer feedback.
Based on their previous studies, Weinberger et al. (2007) proposed these measures to assess a group's knowledge convergence:


Knowledge equivalence score. Each individual member's score of valid knowledge items is counted. A group's knowledge equivalence score is calculated by dividing the standard deviation of its team members' scores by the members' mean score.
 Shared knowledge score. A pair-wise comparison is first conducted to examine the level at which group members use the same valid knowledge items. The score obtained is then divided by the members' mean score, which produces the group's shared knowledge score.
In studies with large sample sizes, the above measures could be used to perform group-level statistical analysis. In this research, due to the small sample size, in addition to computing these two group-level scores, the calculation and use of shared knowledge score were extended, and an individual-level shared knowledge score was also computed for each participant. The generation of this individual-level score allowed further statistical analysis in order to verify the occurrence of knowledge convergence through reciprocal peer feedback. Although this individual-level score could also be applied to studies with large sample sizes, the introduction of this measure makes it possible to statistically inspect the occurrence of knowledge convergence in smaller classes.

Theoretical rationale
In this study, after creating a concept map indicating high school students' possible solutions to a mathematics problem, pre-service teachers were asked to engage in an online peer feedback activity: Participants shared their individual maps with the other four team members, commented on their members' concept maps, and then responded to suggestions from their peers. It was hypothesized that such a process would enhance learners' knowledge convergence. The following paragraphs elaborate on the rationale of this research.
Studies have shown that assessors learned from providing feedback to their peers. For instance, Li et al. (2010) found that undergraduate teacher education students who had offered feedback of higher quality to their peers also created better projects. Cho and Cho (2011) asked undergraduates in a physics course to review peers' writing assignments. Both the assessors who had provided more comments that discussed strengths concerning the content of multiple paragraphs and those who had pointed out more issues regarding the content of a paragraph in peers' writing tended to submit revisions of better quality. In Lu and Law's (2012) study, assessors who had shared more suggestions and comments discussing possible areas of improvement in peers' project performed better in their own final projects. Therefore, the author argues that pre-service teachers in this study might learn from their peers in the process of providing feedback. Specifically, the process of reviewing peers' maps might increase participants' awareness of their peers' ideas (Engelmann & Hesse, 2010). Pre-service teachers also had access to other teammates' feedback for the same peer's map that they commented upon, highlighting strengths and potential areas of improvements. Both of these processes might facilitate the occurrence of observational learning (Bandura, 2003), prompting the assessors to compare peers' maps with their own and to incorporate what they had learned from that observation into their own maps. Moreover, in order to provide feedback to their peers, pre-service teachers needed to articulate their thoughts (Liu & Carless, 2006). Through self-explanation, they might be able to identify missing information in their own maps or their misconceptions, which could also improve their understanding about the topic (Coleman, 1998).
Additionally, receiving feedback from peers improves the learning of assessees. For instance, in the study by Cho et al. (2008), getting feedback from multiple peers more effectively enhanced the quality of recipients' writing than receiving comments from experts. Xiao and Lucking (2008) compared the results of providing learners with both rating grades and feedback from peers versus only offering them peers' rating scores, and they found that the former practice better promoted improvement in the students' writing. Feedback from peers prompts recipients to engage in self-assessment, identifying gaps in their knowledge and reflecting on what can be done to enhance their learning product (Liu & Carless, 2006). The fact that peers might not always be right might encourage assessees to engage in "mindful reception" of peer's views as they look for information to verify or reject peers' opinions (Gielen, Peeters et al., 2010). Additionally, when different peers express conflicting suggestions or when peers' opinions contradict one's own, cognitive disequilibrium (Kibler, 2011) might be triggered, prompting assessees to actively resolve the disagreements. Such a process of resolving discrepancies might further facilitate one's building upon peers' ideas (Weinberger & Fischer, 2006). Also, having to elucidate whether or not actions would be taken based on assessors' feedback might further promote participants' mindful reception of the suggestions from their peers. For instance, Gielen, Tops, Dochy, Onghena, and Smeets (2010) found that after receiving peer's feedback, students who were asked to provide a response explaining the revisions that were performed based on peer's proposals improved more in their writing than those who were not asked to do so. In this project, assessees were asked to reply to their team members' comments, explaining why they might include or reject the suggestions of peers into their revision. This practice not only closed the feedback loop, but it also might further encourage the assessees to evaluate their own maps and to interact with peers' ideas during their revision.
In summary, it was argued that both providing feedback on peers' maps and also receiving and responding to feedback from peers could facilitate pre-service teachers to identify strengths and weaknesses in their individual maps and to subsequently include peers' ideas into their maps. It was hypothesized that this process would encourage the occurrence of knowledge convergence.
Additionally, two explorative investigations were conducted. First of all, the potential influence of participants' interaction process on their knowledge convergence was explored. Previous research (Barron, 2003;Fischer & Mandl, 2005;Jeong & Chi, 2007;Roschelle, 1992) has demonstrated the impact of interaction on groups' knowledge convergence. For instance, Fischer and Mandl (2005) asked educational science students to read a text about an educational theory. The students then drew concept maps in dyads to prepare spoken evaluations for three lesson plans using this theory. Individual pre-and post-tests were administered. The researchers observed that dyads successful in knowledge convergence had shorter conversational turns during discussion and more frequently attempted to build upon prior contributions. Therefore, this study also investigated the three groups' discussion processes and looked for possible differences among them. Furthermore, pre-service teachers' knowledge convergence scores of different concepts were compared. For instance, in the above mentioned study by Fischer and Mandl, open-ended questions were employed to examine learners' factual knowledge. To test learners' application of their understanding, Fischer and Mandl also asked learners to provide an oral evaluation of a new case. It was found that the occurrence of convergence was more prominent in the tasks that required learners' application of what they had learned, compared with their convergence of factual knowledge. Hence, this study also explored participants' convergence in recognizing different concepts involved in mathematics problem solving. The findings might be informative for practitioners interested in cultivating learners' knowledge convergence.

Participants
Fifteen pre-service mathematics teachers participated in this study. They were taking an undergraduate course on instructional methods of secondary school mathematics at a large public university in the Midwest. They all had field experiences teaching mathematics in secondary schools. According to Table 1, the majority of them (14 out of 15) were in their third-year of college. The male to female ratio was 7 to 8, and their average age was approximately 20.

Procedures
Before this project began, basic concepts and skills necessary to create a concept map were introduced to the pre-service teachers, and they were asked to practice constructing a concept map individually using CmapTools. Feedback was provided for each concept map.
After that, these 15 pre-service teachers were divided into three groups of five: Blue, Green, and Red Teams. An algebra problem was provided: "John bought a certain number of apples at 30 cents each and he had 3 dollars left. If instead, apples were 40 cents each, he would have been short 1 dollar. How many apples did he buy? Show your work." Also available were six examples of secondary school students' works (see Fig.1). The pre-service teachers were asked to first solve the algebra problem themselves and then analyze students' works. After that, they individually created a concept map by identifying the key words in the possible solutions to the algebra problem and explaining the relationships among these concepts. Then they shared maps with their teammates and commented on each member's concept map online, addressing both peers' strong points and areas that needed improvement. Additionally, the pre-service teachers were expected to reply to the comments that they had received, stating whether or not they agreed with their team members' feedback and why. After this online discussion, they revised and resubmitted their own maps.

Fig. 1. Examples of student works
Moreover, individual knowledge examinations took place before and after the pre-service teachers had engaged in the peer feedback process. Both the pre-and postknowledge tests included the same algebra problem as above but with different examples of student works. The post-test also contained a new but similar problem accompanied by one example of secondary school student work. Pre-service teachers were asked to analyze the general mathematics knowledge necessary to solve the problem, provide diagnosis of student works, and discuss possible feedback for their students (see Table 2 for more details).

Table 2
Sample knowledge test materials

Description Samples
New problem Nanda has a tall, thin candle and a short, thick candle. The tall, thin candle is 40 centimeters tall. It loses 3 centimeters in height for each hour it burns. The short, thick candle is 15 centimeters tall. It loses one centimeter in height for each hour that it burns. Nanda thinks that if the tall, thin candle and the short, thick candle are lit at the same time and allowed to burn continuously, at one point in time they will be exactly the same height. Is Nanda correct? If your answer is yes, tell when the two candles will be the same height. If your answer is no, explain why the two candles will never be the same height.
Questions  What concepts do 9th grade students need to know to solve this problem?
 What does the student understand and/or what understanding is lacking? Explain your answer.
 What questions would you ask to examine the student's understanding further? Justify your answer.

Data coding
In order to analyze pre-service teachers' learning performances, their responses in the pre-and post-knowledge tests, their 15 individual concept maps created before the peer feedback process, and their 15 revised maps were analyzed. However, due to the fact that two Blue Team members did not participate in the knowledge tests, participants' maps were used as the main source of data representing their learning outcomes. Analysis of knowledge exam responses was still conducted, but the result was mainly adopted for purposes of triangulation.
A coding scheme to categorize the concepts that the pre-service teachers had used in their maps was derived both inductively and deductively by two investigators. One researcher had extensive experiences studying knowledge construction and concept mapping. The other has been a secondary school mathematics teacher since 1994 and was then working on a Ph.D. program of mathematics education. A review of the 30 maps was conducted, and a list of key concepts identified by the course instructor guided the construction of the coding scheme. Additionally, Gick (1986) and Jonassen (1997) analyzed the major stages involved in the solving of well-structured problems, including W. Chen (2017) building a representation of a problem, searching and crafting solutions, and finally carrying out a solution. Their works also guided the creation of the coding scheme. Eventually, a coding scheme that involved 25 concept categories was constructed and was utilized to analyze the maps. In Table 3, these concept categories were organized into eight higher, second-level and 17 lower, third-level concept categories. During coding, each concept was placed into the most specific, lowest possible category. Three top firstlevel concept categories were also added into the coding scheme for organizing purposes, but they were not adopted for the coding. Pre-service teachers' responses to the knowledge questions were coded by two investigators who had extensive experiences researching concept mapping and knowledge construction. One of them also served as a teaching assistant for this course. Key concepts and categories needed to solve these algebra problems were also identified by the instructor. They were used to guide a review of the participants' responses. After that, the two researchers developed a coding scheme (see Table 4) that included seven concept categories and analyzed all the responses.

Table 4
General knowledge test coding scheme

Category
Description System of equations Solving two equations that involve the same set of variables

Guess and check
Guessing a possible answer and checking whether the answer is correct Finding an intersection between two lines in a graph Arithmetic Basic arithmetic knowledge for applying mathematical strategies

Mathematical representation
Transforming a word problem with mathematical symbols Problem situation Common knowledge needed to understand a problem situation (e.g., relationships between dollars and cents) In order to study pre-service teachers' online interaction, their discussion board messages were coded. The same investigators who had analyzed the knowledge test results also segmented the participants' online discussion board messages and identified emerging themes. The grounded theory approach (Strauss & Corbin, 1990) was applied. A category structure was developed deductively through continuous negotiation between the two investigators and dynamic interaction between the text and the researchers. In Table 5, the resulting coding scheme included categories of messages that pointed out strengths in members' maps, detected issues and offered suggestions for improvement, and also responded to peers' feedback. This coding scheme was utilized to analyze the messages. Pointing out problem(s), asking for clarification, or providing suggestion(s) relating to syntax, typos, and wording of the concepts and links I think the relationship between graphs and intersection points is mean to be "by looking at," and not "by loking at."

Annotations
Suggesting adding or changing annotations …Add annotations! You are graded on this… Organization Pointing out problem(s) or providing suggestion(s) relating to the organization of the map …My opinion would do develop a concept map with the methods on how to solve the problem first then from the different methods state which student used the approach rather than starting with the students method. Formatting styles Pointing out problem(s), asking for clarification, or providing suggestion(s) regarding the formatting styles, including font sizes, line styles, and use of colors.
…The only thing I might suggest is add some bright colors to direct your attention towards the starting point of your concept map…

Response
Accepting comments Expressing an appreciation or acceptance of peer's suggestion(s) I agree! Thanks for the input.

Revision
Reporting changes performed or will be done in response to peers' suggestion(s) I will be sure to add some annotations explaining this.

Defending a position
Expressing disagreement with peer's opinion(s), sometimes with justification provided I'm not sure that I see a benefit of forming sentences with my linking phrases.

Knowledge convergence scores
Calculation of knowledge equivalence score. Firstly, the total number of concept categories used by each team members was counted. After that, the standard deviation and the mean of five team members' scores were computed. Dividing the standard deviation of the group members' scores by the average of their scores yielded this group's knowledge equivalence score.
Calculation of shared knowledge score. Following the method proposed by Weinberger et al. (2007), group-level shared knowledge score was computed. Analysis of participants' concept maps and knowledge test responses followed these same steps detailed below: Step-a1: Pair-wise comparisons among five group members' learning products were conducted for each of the concept categories listed in the coding scheme. A score was given to each concept category following this procedure:


Ten was credited for a concept category when all five members shared this category in their individual learning products,  six was granted when altogether four members had included this category in their learning products,  three was credited when three members had used this category,  one was given when only two members had adopted this category, while  zero was credited when fewer than two members had mentioned the category.
Step-a2: Scores for all the categories were added together.
Step-a3: The sum obtained in Step-a2 was divided by the team members' average score (see calculation of knowledge equivalence score) in order to compute this group's shared knowledge score.
Next, Weinberger et al's (2007) procedure of group-level shared knowledge score computation was extended, and an individual-level shared knowledge score was calculated for each pre-service teacher. Due to the existence of missing data, only their concept maps were included in this analysis, and the steps were listed below: Step-b1: The same Step-a1 for the calculation of group-level shared knowledge scores was performed.
Step-b2: Scores for the categories that a pre-service teacher had used were added together.
Step-b3: The total number of unique concept categories that all five team members had used was also calculated.
Step-b4: Due to the fact that a group that had mentioned more categories would have a higher chance of sharing more categories, thus increasing each member's total score obtained in Step-b2, a participant's total score was divided by the total number of categories that was obtained in Step-b3, yielding an individual-level shared knowledge score. Table 6 presents each team's group-level knowledge convergence scores. According to the definitions by Weinberger et al. (2007), higher shared knowledge and lower knowledge equivalence scores indicate that a team have more success in knowledge convergence; while lower shared knowledge and higher knowledge equivalence scores reflect the opposite. Based on an analysis of the 30 concept maps, the Green Team demonstrated the greatest amounts of improvement in both shared knowledge (2.14) and knowledge equivalence (0.23) scores. In contrast, the Red Team had the lowest amount of increase in their shared knowledge score (0.49). Their performances even became less equivalent after the activity, as the difference between their post-and pre-knowledge equivalence scores was positive. Due to the existence of missing data, analysis of preservice teachers' knowledge exam responses was only conducted with the Green and Red Teams. According to Table  6, the Green Team achieved more improvement with both shared knowledge (1.31 vs. 0.86) and knowledge equivalence (0.10 vs. 0.09) scores compared to the Red Team. This finding was consistent with the results of concept map analysis.

Pre-and post-comparisons
Analysis of individual shared knowledge scores was conducted with concept maps. Participants' individual scores prior to the feedback activity ranged from .83 to 2.65, averaging 2.12; their scores post the activity ranged from 1.48 to 5.15, averaging 3.60. Similar to the interpretation of group-level shared knowledge scores, an increase (M=1.48) in a pre-service teacher's individual shared knowledge score after the peer feedback process indicated that the participant's understanding became more similar to his or her team members through the process.
Next, pre-service teachers' preand post-individual shared knowledge scores were compared using SAS 9.4 software. Repeated ANOVA with errors clustered under groups was performed. According to the results, their individual shared knowledge scores increased significantly after the peer feedback activity, F (1, 26) = 30.46, ρ< .0001. Also, 25% of the variability in their individual scores could be explained by grouping. The average increase in the pre-service teachers' scores after they had engaged in the peer feedback activity was 1.48, and it fell within the 95% confidence level [.93, 2.03]. Table 7 lists the result from the coding of discussion board messages. During online discussion, pre-service teachers provided comments on various aspects of peers' maps, including the content of concepts and links, annotations, and map organization. Some of them also replied to the proposed changes from their peers and explained the revision that was performed based on peers' comments or their rejection of peers' suggestions. Compared with participants of the other two groups, the Red Team composed more messages (7 vs. 1 & 2) defending for their own ideas in response to peers' comments, indicating that they more often rejected integration of others' perspectives. Also, the Red Team had more messages that provided suggestions about syntax, wording, and typo problems (7 vs. 1 & 2) than the Blue and Green Teams. Additionally, a word count of discussion board messages was conducted. It was noted that the Red Team members (47.76) wrote shorter messages than members of the other two teams (70.18 and 62.30, respectively).

Convergence of different mathematics concepts
In order to explore possible differences in pre-service teachers' sharing of various major mathematics concept categories, group-level shared knowledge scores were computed for the eight second-level concept categories listed in the coding scheme (see Table 3). In other words, for this exploration, when a concept was originally coded under a third-level concept category, it was treated here as its affiliated higher-level, more general category. A score was obtained for each major concept category following the procedure listed in Step-a1 (see Section 3.3.2). For each category, the scores earned by the three teams were then averaged. Table 8 presents the result of this preliminary exploration. It was noted that the amount of improvement in the participants' convergence of concepts related to problemsolving strategies was greater than that for concepts representing problem context and domains. Specifically, the amount of improvement in their group-level shared knowledge scores of problem context and domain concepts (categories from 1.1. to 2.2 in Table 8) ranged from .00 to 1.00; in contrast, their increase in the scores of problem-solving strategies concepts (categories from 3.1 to 3.4) was greater, ranging from 1.00 to 4.67. It was also noted that prior to the peer feedback process, participants' average shared knowledge scores of algebra (10.00), arithmetic (7.67), and solving with equation (7.67) were much higher than their scores of solving with a table (1.00), a graph (3.00), and guess and check (3.00) and also concepts (categories of 1.1 and 1.2) relating to problem context (3.33 and 3.67). Although pre-service teachers were able to benefit from peer feedback by enhancing their sharing of concepts relevant to solving the problem with a table, a graph, and guess and check (with increases ranging from 3.67 to 4.67), there was little or no increase (0.00 and 0.33) in participants' convergence in the concepts about problem context. To further explore this area of research, pre-service teachers' individual shared knowledge scores regarding the concepts belonging to the categories under problem context and domains and their scores relating to categories of problem-solving strategies were computed, following Steps-b1 to b4 (see Section 3.3.2).


For the concepts about problem context and domains, participants' individual scores prior to the feedback activity ranged from 1.25 to 3.60, averaging 2.63; after the peer feedback process, their scores ranged from 1.00 to 6.25, averaging 3.61, with an increase of 0.98.


Regarding the concepts about problem-solving strategies, participants' previous scores ranged from 0.00 to 2.78, averaging 1.36; their individual scores post the activity ranged from 0.64 to 5.40, averaging 3.44, with an increase of 2.08.
Pre-service teachers' individual scores before and after the peer feedback process were compared using repeated ANOVA with errors clustered under groups. The scores representing participants' sharing of concept categories about problem context and domains in their maps were first analyzed. Although there was an increase in the average of their individual shared knowledge scores after the peer feedback process, this difference was only marginally significant, F (1, 26) = 7.48, ρ= .011. In contrast, the individual scores representing pre-service teachers' sharing of concept categories relating to problem-solving strategies increased significantly after the online activity, F (1, 26) = 55.53, ρ< .0001.

Discussion and conclusion
Through comparisons of pre-service mathematics teachers' preand post-learning products, this study was able to demonstrate that team members' knowledge converged through the online peer feedback process. In this study, the approach proposed by Weinberger et al. (2007) was successfully extended by computing individual-level knowledge convergence scores for statistical analysis. Adoption of this measure also makes possible future investigation analyzing the impact of other individual-level factors on team members' knowledge convergence.
Several limitations did hinder the scope of this initial research exploring the impact of peer feedback on knowledge convergence. For instance, due to practical constraints, it was not possible to deploy a control group in the current project. Future studies could be conducted comparing experimental and control groups. Additionally, evaluation methods of concept maps abound, and different researchers have analyzed maps on the concept, proposition, structure, and map levels, respectively (Jonassen, 2006;Novak & Gowin, 1984). However, the latter two approaches were found to have low validity and reliability (McClure, Sonak, & Suen, 1999). In this exploratory study, the analysis of concept maps focused on the participations' convergence of concepts. Other researchers might also be interested in inspecting team members' convergence of propositions in the future. The author of this study is currently conducting another research to explore a method for such an analysis.
Several follow-up investigations could also be performed to expand the understanding acquired through this research. Currently, studies have been conducted analyzing the influence of different instructional interventions on students' learning from the peer feedback process, including asking learners to specify areas that need feedback from peers or to explain actions taken based on peers' feedback (Gielen, Peeters et al., 2010;Gielen, Tops et al., 2010). In the future, researchers might also be interested in understanding the impact of these interventions on the knowledge convergence of team members participating in a peer feedback activity. It was also noted that studies on the effect of an intervention on participants' short-term learning achievements through the peer feedback process and studies focusing on long-term outcomes of the same intervention sometimes reached conflicting conclusions (Gielen, Peeters et al., 2010;Gielen, Tops et al., 2010). This study only examined team members' short-term learning outcomes. It might be worthwhile to also investigate participants' long-term knowledge convergence after the peer feedback process.
Additionally, the analysis of interaction messages indicated that, compared with the other two teams, the Red Team members who converged less knowledge rejected peers' comments more frequently, although they exchanged more messages during the online discussion. Consistently, Barron (2003) found that more successful teams tended to accept or discuss correct proposals, while unsuccessful groups tended to reject or ignore correct proposals suggested by their members. In addition, the Red Team members seemed to focus more on identifying minor issues in peers' maps. While making attempts to help peers improve the content of their maps might encourage team members to share understanding of essential concepts, discussion about language and formatting problems was less helpful. It was also noticed that the messages exchanged among the Red Team members were much shorter than the messages from other teams. This seemed to contradict findings from Fischer and Mandl (2005), who found that dyads that were more successful in achieving knowledge convergence uttered relatively shorter turns than pairs that were less successful in attaining knowledge convergence. However, Fischer and Mandl's study engaged students in synchronous interaction, while pre-service teachers in this study participated in asynchronous discussion. Future studies about the influence of feedback characteristics and interaction patterns are necessary during exploring participants' achievement of knowledge convergence through peer feedback.
It was also found that different mathematics concepts were not equally shared among group members. After giving and receiving comments, pre-service mathematics teachers improved most in their convergence of concepts relating to problem-solving strategies than in other concepts relevant to problem representations. They especially benefitted by the peer feedback process in acquiring more atypical strategies to solve the algebra problem, including solving the problem by using a table, a graph, and guess and check, and they successfully integrated these ideas into their revisions. Prior to the peer feedback activity, their convergence on these concepts relating to atypical problemsolving strategies and other concepts representing problem context were lower than their convergence on the concepts about problem domains, although their improvement in sharing ideas relevant to the problem context was limited after the online discussion. These findings supplemented the understanding obtained from Fischer and Mandl's (2005) study comparing learners' convergence of factual and application-oriented knowledge. Future research is also recommended to investigate why some concepts were more easily shared through peer feedback than others and to compare the impact of different instructional activities and interventions on participants' sharing of different types of concepts. Findings from these explorations might be helpful for practitioners who are interested in engaging learners in collaborative activities and in encouraging them to learn different concepts from their peers.