Suggestions for teacher education from concept mapping studies

In order to enhance primary and secondary education, teaching and learning methods need to be continuously developed as well as, of course, promote teaching quality dependent on teacher personality, teacher professional development, teacher self-development, etc. Teacher professional development gives the novice teacher access to a wide set of teaching methods and assessment opportunities, especially geared to flexible learning and assessment methods, which can be considered for adoption. One such flexible method is the use of concept mapping. This article describes the results of several studies, where concept mapping method was used, giving many didactical suggestions for using concept mapping for learning and especially for assessment. Additionally, considerations are introduced on using concept maps as a research instrument.


Introduction
Teacher education is expected to enable teachers to acquire different competences, including the capability to develop and use learning and assessment methods.For example, teachers of natural science need to enhance students' meaningful learning and scientific literacy through using interdisciplinary approach, visualization techniques, etc.One issue with the mentioned student attributes is that acquisition or attainment are difficult to assess.This article gives suggestions for teachers and teacher educators related to concept mapping, based on the results of several studies that have been carried out using the concept mapping method for assessment (Fig. 1).
Teacher professional development should promote the novice teacher's ability to manage a wide set of teaching and assessment methods.Nowadays, the emphasis of teaching and assessing has shifted towards supporting and directing students in their school studies so that they are able to integrate meaningfully the knowledge gained with their everyday life; that they are able to gain tacit knowledge: "how to be and what to do" (Roegge, Wentling, & Bragg, 1996;Ó lafsdóttir, 2011).
To reach to the above-mentioned direction, constructivist principles and teaching approaches are important so that learning occurs when students are involved in a process of meaning and knowledge construction.Each student constructs his or her own concepts or knowledge (Novak, 2010).Ausubel (1968) recognises the need within constructivism to separate rote memorization and meaningful learning (Emenike, Danielson, & Bretz, 2011).Bretz (2001), highlighting the need for meaningful learning, wrote that according to constructivist ideas, meaningful learning only occur when education provides experiences that require students to connect knowledge across three specific domains.Meaningful learning takes place, when: 1).The student has relevant prior knowledge that integrates with the studied knowledge ; 2). the learning materials are meaningful in and of themselves, concepts acquired and propositions understandable; and 3). the student chooses to assimilate this meaningful learning into his/her existing knowledge.

Fig. 1. Some competences and skills that are needed in teacher training
As illustrated in Fig. 1, teacher education is expected to provide different set of methods for the novice teacher.Many of these methods could be assessed and also improved through the use of a concept mapping method.But still there are few studies about assessing with concept mapping.Especially those studies, where the results of concept maps would have compared with some other assessment method.

Scientific literacy and interdisciplinarity
It is difficult to clearly define concepts such as scientific literacy and interdisciplinarity, even though they are very familiar terms.
Scientific literacy, as a competence, is essential for everyday life.However, it is very difficult to assess this.Holbrook and Rannikmäe (2009) defined scientific literacy as an ability to creatively utilize appropriate evidence-based scientific knowledge and skills, particularly with relevance for everyday life and a career, in solving personally challenging yet meaningful scientific problems as well as making responsible decisions.In a publication in 1993, Biological Science Curriculum Studies (BSCS) (1993) and further by Bybee (1997), divided scientific literacy into four different levels based on the complexity of different skills and abilities.These levels were called: a) nominal literacy; b) functional literacy; c) conceptual and procedural literacy and d) multidimensional literacy (Abrams, Southerland, & Silva, 2008, p. 29).The highest levelmultidimensional literacy -assumed also students' interdisciplinary knowledge, otherwise it would be impossible to create links that were across scientific disciplines and to understand the nature of science, the history of science and the role of science in personal, social and global life and to connect different disciplines with technology, society and each other (Soobard & Rannikmäe, 2011).
Nicolescu defines i n 1997 interdisciplinarity as the transfer of methods from one discipline to another (Dillon, 2008).In this study, the preferred definition for interdisciplinary is: the ability to integrate knowledge and thinking in two or more disciplines or established areas (Mansilla & Duraisingh, 2007).Teachers can foster students learning interdisciplinary by activating their prior knowledge and by helping to create links between different concepts and to their previous knowledge and understanding.
Different studies have shown that interdisciplinary teaching was useful for students.Mansilla and Duraisingh (2007) noted that interdisciplinary learners were able to integrate information, concepts, theories, etc. from two or more disciplines.In 2002, Ivanitskaya and her research team summarized identified outcomes of an interdisciplinary approach, which included: changes in attitude; cognitive skills development; motivation of students for deep learning; creation of personal understanding; development of critical thinking; ability to plan, monitor and control learning.Other authors have pointed out that only a few studies at the college level have indicated that interdisciplinary studies had positive effects on learning (Lattuca, Voigt, & Fath, 2004), which have pointed to the need for further assessment and research.Ivanitskaya, Clark, Montgomery, and Primeau (2002) have indicated that the assessment of cognitive outcomes of interdisciplinary learning should give teachers an insight into students' development.Stowe and Eder (2002) agreed that multiple sources of data should be used for interdisciplinary assessment (tests and surveys among them).
In 2007, Mansilla and Duraisingh pointed out that there is lack of clarity related to how to define indicators of quality interdisciplinary work.Klein (2002) pointed out that interdisciplinarity is a complex idea, one that is not readily subjected to the seemingly reductive processes of assessment (Stowe & Eder, 2002).For that, a flexible assessing method was needed, because often the ordinary open-ended or multiple questions tests only pointed out the rote-learned facts and did not enable assessment of concept acquisition, or understanding of principles.For such cases, one possibility for assessment was the use of concept mapping (Schaal, Bogner, & Girwidz, 2010).

Assessment
One important teacher task was seen as the ability to give feedback to students on their learning and development.Rowntree (1987) described assessment as giving value on something.We could assess different aspects of knowledge (facts, principles), beliefs, skills etc. in education.Klassen (2006, p. 821) wrote: "The literature on assessment reveals that greater reflection and discrimination between assessment practices are warranted to make improvements in this area.According to cognitive psychology, understanding is a mental process of perceiving and knowing."Assessment (a) can be used to give information about one person, or a group of persons; (b) outcomes are often given as marks and grades; (c) can be undertaken using many different assessment methods.One of the most common and used is testing (Novak, 2010;Rowntree, 1987).
Although there are different types of assessment methods, such as multiply choice test, open ended test, quizzes etc. there are also arguments about what a test is actually measuring.Wiggins (1989) points out that a test does not merely control "standardized" work in mechanical way, but also depends on the test creation and student's capability.At the same time, testing is an overused assessment method and also other assessment methods need to be developed and used (Novak, 2010;Little, Bjork, Bjork, & Angello, 2012).These new assessing methods need to be flexible, on-going and give feedback for supporting learning (Stowe & Eder, 2002;Borrego, Newswander, McNair, McGinnis, & Paretti, 2009;Schaal, Bogner, & Girwidz, 2010;Mansilla & Duraisingh, 2007).

Concept mapping for assessment
Joseph Novak and his research team initiated the concept mapping method, based on the meaningful learning theory (Ausubel, 1968).They indicated that a concept map was a collection of propositions (one proposition consisted of 2 concepts and could be "read" as a sentence) that expressed personal understanding.Novak (2010) pointed out that nowadays, it was easy to create and assess concept maps using a special computer program and thus concept mapping method could be used meaningfully for assessment (Novak, 2010;Cañas, Novak, & Reiska, 2014;Klassen, 2006).
Nevertheless, problems have occurred when using concept mapping as an assessment method as there are many different measures and values that could be taken into account.Researcher or teachers were required to decide the purpose of the assessment and what kind of instructions was given to students.Schwendimann (2014) pointed out quantitative measures like: concepts, hierarchy levels, propositions and the structure of network were different for different assessment tools, whereas some scientists had also taken into account the diameter of a concept map.In 1995, Austin and Shore (Schwendimann, 2014) pointed out that a higher number of links did not guarantee that student understood the topic better, as many links could be invalid or trivial.Furthermore, for describing semantic changes between concepts, there was a need for quality measures.Cañas, Novak and Reiska wrote in 2014 (p 269): " …because of the nature of the work, evaluation of the quality of concept maps in other applications is not undertaken in a formal way as in education".They explained that concept maps consisted of a structure and content.Assessing the quality of a concept map needed expert judgement.Sometimes concept maps could be assessed in comparison with experts' concept maps (Cañas, Novak, & Reiska, 2014), while in other cases, experts could rate the propositions.Schwendimann (2014) pointed out that it was also possible to assess the topology (geometrical structure) of the concept map, or types of propositions.

Large-scale study in Estonia
A large-scale study and several case studies about science teaching at high school level were carried out in Estonia from 2009 to 2014 (Soobard & Rannikmäe, 2014).Altogether, 1614 10 th grade students participated from different schools in Estonia (Soika & Reiska, 2013).This paper focused on 343 students who answered the same PISA-like test topic and also the focus question in undertaking the concept mapping.This allowed a direct comparison of results.
The aim of the overall study was to determinate aspects of scientific literacy and the interdisciplinary knowledge of students.In line with comments by Stowe and Eder (2002) data were collected through concept mapping and multi-dimensional, PISA-like tests.
Students were separated into four groups by solving different three-dimensional PISA-like exercises.Most students created also a concept map related to the topic for one PISA-like exercise.The focus question for creating a concept map was: "Instant cold pack -is it only chemistry?"Students could utilise 30 pre-given concepts from different disciplines (identified as chemistry, physics, biology and everyday life).The categorisation of concepts into the different disciplines was undertaken by 85 experts (university and high school teachers) (Soika & Reiska, 2014b;Soika & Reiska, 2014d).

Case studies
During 2009-2011, four different case studies were developed to research how concept maps could be used for assessment.Concept mapping was used as a research instrument, because it was seen as giving a unique opportunity to illustrate the creation and changes of structure of concepts in students' knowledge and understanding, thus allowing more information to be gained about concept mapping as an assessment tool.
Each case study included approximately 50 students, aged 15-17.Besides the main purpose, other components that were included were based on the learning material, teaching method, or student comprehension (Soika, Reiska, & Mikser, 2010;Soika & Reiska, 2014a;Soika & Reiska, 2014c).The content of the concept maps was based on an abstract chemistry-related topic, or on the solving of a scientific exercise.
These studies were similar in their structure.Concepts were pre-given and students created topic-based concept maps.In every study, students created two different concept maps.The first determined the level of students' prior knowledge, while the second determined changes in students' knowledge that had taken place during lessons, or after a specified period.Every time (before and after learning), students also answered a knowledge test that contained different exercises.
Results of the quality of concept maps' proposition were assessed manually by experts and the main parameters (such as taxonomy score, orphan count, proposition count etc.) were analysed by a computer-based program.Results of the pre-and poststudy concept maps and knowledge tests were compared.In this paper, an assembled overview of the results was undertaken, seen as useful for teachers or teacher educators when working with concept mapping (Soika, Reiska, & Mikser, 2010;Soika & Reiska, 2014a;Soika & Reiska, 2014c).

Assessment of the maps in the studies
In the studies described in this paper, the concept maps were made using the computer program -CmapTools.Different parameters were calculated with Cmapanalysis (Cañas, Bunch, Novak, & Reiska, 2013) and MS Excel.
Cmap-analyse was used to obtain the main and general data from the concept maps.Rating was made manually, pointing out right and wrong propositions.As students also completed three dimensional PISA-like tests, the results of concept maps-given ratings were analysed and correlated for comparison with the PISA-like tests.
Propositions were rated independently by two experts, each awarding rates based on: 2-very good subject-based proposition; 1-correct ordinary daily-used proposition, or subject-based proposition with some queries; 0-wrong or unclear proposition.Different experts evaluated the proposition quality and concepts clusters.Some explanations of the measures used in the studies are given (Cañas, Bunch, Novak, & Reiska, 2013;Novak, 2010):  proposition count (the number of propositions ("sentences"),  proposition quality (expert given rating to propositions-the aim is to select propositions by the subject based quality);  branch point count (the total number of concepts and linking phrases that has at least three connections);  taxonomy score (numerical value for the map that considers different quality and quantity measures of concept map),  intra-cluster proposition count ("sentence", that is created from concepts from the same cluster (discipline));  inter-cluster proposition count ("sentence", that is created from concepts between different clusters (disciplines)).
This article does not elaborate on the process of analysis of these studies, but emphasizes the results and outcomes connected to the concept mapping method.Suggestions for teacher professional development are also given, based on these studies.The summary of the results and suggestions is presented in Fig. 2.

Suggestions for teacher education
There are several possibilities for using concept mapping as an assessment tool, but there are also problems, which need to be solved.The following describes the experience of assessment with concept mapping and provides suggestions based on the results of the studies and also based on an analysis of the literature.The suggestions for teachers, teacher educators and researchers are divided into two sections: a) How to use concept mapping as an assessment tool (Chapter 4.1).b) How to improve science teaching (Chapter 4.2).

Suggestions for using concept mapping as an assessment tool
A concept map can reflect the structure of learning material, or the methodology that the teacher uses (Soika, Reiska, & Mikser, 2010;Soika & Reiska, 2014a).This method is very flexible, but meaningful.When it is used for assessment, teachers define the important outputs to be assessed.Instructors analyse whether a focus question is needed and whether concepts, or even the structure of the concept maps, needs to be given.It is also important, where and when the concept mapping is undertaken, because students' cognition depends on several factors, such as students' mood, classroom comfort and instruction given.It needs to be noted that concept maps constructed in very different environments and situations, are not comparable (Soika & Reiska, 2013).

General suggestions on creating and using concept mapping for assessment
A. Concept mapping, as an assessment method, is valid and reliable, but only under certain circumstances.Analyses of studies show that the results of concept mapping and PISA-like multidimensional tests are comparable.Concept mapping can point out students' understandings more specifically than any other test (Soika & Reiska, 2014b).B.
Concept mapping is a flexible assessing instrument and can therefore be used more often.For easier assessing and creating, concept maps can be generated with computers.As a computer is a common tool for students, research indicates that teachers need not be afraid that students get confused with concept mapping because of the computer.Concept maps constructed using computers are also easier for teachers to assess, because there are flexible assessing possibilities available (Soika & Reiska, 2014b).This is especially important for large-scale studies (Soika & Reiska, 2013).C. When comparing concept maps, concepts should be pre-given to students, before creating concept maps.Also, within a limited timeframe, students could then create better structured concept maps.It was found that students were able to create more high scored propositions with pre-given concepts (Soika & Reiska, 2014a).

Suggestions to assess interdisciplinarity with concept mapping
The concept mapping method can be recommended for assessing the interdisciplinarity and scientific literacy levels of students.Schaal, Bogner, and Girwidz ( 2010) and his research team admitted that traditional tests often fail, when interdisciplinary knowledge is assessed.In 2009, Borrego with his team showed that concept mapping is an assessment method that can be used for assessing interdisciplinarity.So far, very few studies have been carried out, where concept maps are used for assessing interdisciplinary approach.

Fig. 3. An example of a poor concept map
Concept maps allow researchers to assess levels of interdisciplinarity and also scientific literacy, because concept maps can often better reflect the structure of students' understanding than standard tests (Soika & Reiska, 2014d).Illustrated below are examples of two concept maps with different interdisciplinary and scientific literacy levels (Fig. 3 and Fig. 4).Fig. 3 shows a poor concept map with low levels of interdisciplinary and scientific cognitive literacy.It is also poorly structured -there is only one branch point, whereas most concept maps create a chain.Although 30 concepts were given to student, the student chose only some of them with most associated with one concept cluster.The propositions in the concept map are not highly subject oriented;-most are from daily use.This student also completed a scientific exercise about concept maps.A comparison of these parameters pointed out that the results from the concept maps and like test statistically correlate.After the experts have separated concepts into subject-based clusters, intra-and inter-cluster propositions appeared (as is illustrated with different colours in Fig. 3 and Fig. 4).
In the assessment of concept maps, the structure of the maps is important.In the studies described, the number of branch points has been taken into account.This measure reflects the structure of the whole concept map.The number of branch points and other structural measures of concept map can be easily analysed using the program, Cmapanalysis.
The above-mentioned parameters are the quantity measures of an interdisciplinary approach.For assessing the quality of a concept map (to give an answer to the question: how well are the propositions structured?), expert opinion and a system of ratings are needed.Propositions can have different qualities, depending on the purpose of the map created.In this study, the following ratings for propositions are used: correct propositions with deep subject background; correct daily-use propositions; incorrect propositions.Proposition ratings give the quality measure of concept maps (Soika & Reiska, 2014d).
The concept map shown in Fig. 4 expresses a high level of interdisciplinarity and also some high level aspects of scientific literacy.Concepts related to physics are blue, chemistry yellow, biology red, and for everyday life green.In the map concepts not only linked within the same cluster, but also with concepts in different clusters.It is possible to notice large differences between concept maps such as when comparing the concept maps in Fig. 3 and Fig. 4, even though the concept maps were created under the same conditions.
Generally, from an analysis of the data, we could conclude that the created concept maps were quite poor, because they contained few propositions.Most propositions were given in the field of everyday life (15,5% from possible ones)-there were connected two concepts from everyday life.Few propositions were inter-cluster, and especially few propositions were created between science and everyday life concepts (as illustrated in Fig. 2).Also were students able to create few 2-scored propositions.
Also calculated was the interdisciplinary quality index (IQI), pointing out the interdisciplinary of the concept map (this took into account the structure and the quality of propositions).The IQI ranged from 1,9 to 0. The average interdisciplinary results was compared with the group-based PISA-like test.The relationship as shown in Fig. 5. indicates the rising trend line comparing results of the PISA-like test and interdisciplinary index.The highest group average IQI reached 0,8.

Fig. 5. Relationship between the interdisciplinary index and PISA-like test
The results from concept mapping were also compared with the results of the multidimensional PISA-like tests.The correlation between the results of the tests and the concept map measures indicated that better results from the test generally linked with more advanced structure and higher rated propositions in the concept maps.Interdisciplinarity and the more cognitive aspects of scientific literacy are related and could be assessed using concept mapping techniques (Soika & Reiska, 2014b).

Suggestions for teachershow to improve science teaching
Below suggestions are given for teachers, based on the results from the studies with concept mapping.
A. It is hard for students to acquire the meaning of abstract concepts (such concepts are often appear as orphans in students' concept maps).To help students to learn more meaningfully, teachers should include more visualization in their teaching and help students in making connections with concepts previous acquired B.
Teachers should engage more in interdisciplinary teaching, as indicated by few students creating concept maps with a high degree of interdisciplinarity and with high proposition ratings (Soika & Reiska, 2014d).When students are able to utilise knowledge from the classroom in everyday life, they enjoy learning more and are probably more interested in the discipline.There are several different possibilities for developing an interdisciplinary approach.For example: solving exercises that combine different disciplines; bringing different examples which combine science with everyday life; using concept mapping to illustration connections between different subject areas, etc. C. The studies pointed out remarkable change between connected concepts right after the learning process and after the summer holiday.Following the principle of Ebbinhaus' forgetting curve (Schacter, 2001), teachers should be aware that there is no such powerful learning method or tool that would help students to remember learned knowledge and skills forever.However, trying to connect new knowledge to an existing one, helps students to build more stable knowledge.D. Teachers should pay more attention to developing students' scientific literacy, because it is needed in their everyday life.For improving and assessing this field, concept mapping method can be used.E.
It would be helpful for teachers if interdisciplinary links were included in the curricula.Teachers need materials that can help them to teach meaningfully using an interdisciplinary approach and with a focus on promoting scientific literacy.

Conclusion
Concept mapping is a powerful method for assessing, teaching, studying, collaborating etc.The studies point out that this method can be used more by teachers and also in the universities, where novice teachers are taught.
Teacher should be able to handle different assessment methods and tools.Especially when they assess, such measures should point out students' gains towards scientific literacy and interdisciplinarity.As an interdisciplinary approach is assumed to be assessed dualistically (Stowe & Eder, 2002), we believe that concept mapping is one appropriate possibility for assessing in this way.
The studies described in this paper highlighted several points for teachers to consider when improving students' interdisciplinarity and scientific literacy levels of learning, and also led to methodological advice about how to use concept mapping as an assessment tool.
The studies also pointed out that the results of concept map based tests correlate with standard knowledge tests and multidimensional PISA-like tests, but that the method by itself is more flexible and in some cases give even more information about gains in knowledge.
When used appropriately, concept mapping is a valid and reliable assessment instrument.

Fig. 4 .
Fig. 4.An example of a concept map with a high degree of interdisciplinarity.Student chose concepts from different clusters and connected them to each other.Many propositions are rated highly.