Developing higher-order thinking skills with concept mapping: A case of pedagogic frailty

Concept maps have been proposed as a tool that can help develop and exercise higher-order thinking skills, including critical thinking, reflective thinking, synthesis, analysis, among others. In this paper, we review how the different aspects of constructing a concept map can help develop and exercise these skills, and discuss why most students never reach the level of concept mapping skills required to exercises their higher-order thinking skills, in what we propose is a case of pedagogic frailty involving the ‘Pedagogy and Discipline’ and ‘Locus of Control’ dimensions.


Introduction
There is general understanding that as time goes by a larger percentage of jobs require employees with higher-order thinking skillsthat is, employees whose work will involve creativity, problem-solving, and critical analysis among other skills (Ananiadou & Claro, 2009;Rimini & Spiezia, 2016).This need results from an ever-increasing interaction with technology, an endless amount of information, and the disappearance of jobs that required repetitive operations and that are being taken over by robots or exported to regions where labor and production costs are lower.Additionally, employees with higher cognitive ability have better professional career opportunities and earn more since jobs that require higher-order thinking skills tend to have higher salaries (Farkas, England, Vicknair, & Kilbourne, 1997;Hall & Farkas, 2011).As a result, the educational systems at all its levels are being pressured to make sure that students develop these higher-order thinking skills.This requires the development of students' capacities to analyze new situations in such a way that their question-asking, problem solving and decision-making is based on a critical thinking framework (Miri, David, & Uri, 2007).Therefore, it is important that educators acquire strategies to assist and help students effectively applying higher order thinking skills to their academic studies (Biggs, 2011;Oliver & Utermohlen, 1995).
In this paper, we explore how concept mapping can be used to develop and promote higher-order thinking skills in students.We begin by reviewing what higherorder thinking skills are, and proceed to discussing how different aspects of the concept mapping process develop and exercise different skills.We conclude by arguing that, in most cases, students don't reach the level of practice and expertise in concept mapping required to develop their critical-thinking skills, due to the lack of time provided by their instructors and the lacking environment in which the mapping takes place.We frame the issue within two dimensions of pedagogic frailty, as proposed by Kinchin and colleagues (Kinchin et al., 2016;Kinchin & Winstone, 2017), as both a disconnection between the practices of the discipline (where concept mapping is the discipline) and the fundamental underlining pedagogy (the 'Pedagogy and Discipline' dimension of pedagogic frailty), and a disconnection between the practicing instructors and the decision-making bodies (the 'Locus of Control' dimension of pedagogic frailty).

Higher-order thinking skills
We don't pretend to provide an extensive analysis or description of higher-order thinking skills, nor do we present a literature review on the subject.We expect the reader to be familiar with the teaching of higher-order thinking skills.Neither are we dwelling with the discussion of the differences between higher-order thinking skills, reflective thinking, critical thinking skills, problem solving, reasoning, and other terms used frequently in the literature and in education (Geertsen, 2003).There is plenty of literature that the reader can refer to on the subject (e.g., Beyer, 1987;Geertsen, 2003;Jonassen, 2000;King, Goodson, & Rohani, 1998;Lewis & Smith, 1993;Pithers & Soden, 2000).We will keep the discussion relevant to concept mapping.
John Dewey (1933) was the first educator who differentiated levels of thinking.He described thinking as a sequenced chaining of events that moves from reflection to inquiry and into critical though processes, and lead to "conclusion that can be substantiated" (p. 5) by more than personal beliefs and images.Thought can "straighten out entanglements, clear obscurities, resolve confusion, unify disparities, answer questions, define problems, solve problems, reach goals, guide inferences, shape predictions, form judgments, support decisions, and end controversies" (King et al., 1998).According to Dewey (1933) thinking does not occur spontaneously but must be "evoked" by "problems and questions"; "To maintain the state of doubt and to carry on systematic and protracted inquiry -these are the essentials of thinking."(Dewey, 1933, p. 14) For this paper, systematic and protracted inquiryquestioning, is a key element in higher-order thinking and concept mapping, as are most of the skills discussed in this section.Newmann (1988) reminds us that higher-order thinking implies a challenge and expanded use of the mind, while lower order thinking takes place during routine, mechanistic applications and constraints on the mind.He continues stating that the challenging and expanded use of the mind is achieved through interpreting, analyzing and manipulating information to solve a problem, because problems cannot be resolved through mechanistic application of previously learned knowledge.It's pertinent to add the importance of the ability to recognize the context of the situation in the ability to recognize, and apply and elaborate knowledge for that situation (King et al., 1998).Transfer is the higher-order thinking skill of being able to apply acquired knowledge to new situations, or new contexts, including across knowledge domains (Barak, David, & Uri, 2007;Zohar & Dori, 2003) Anderson and Krathwohl (2001, p. 63) define transfer in how it differs from retention: "Two of the most important educational goals are to promote retention and to promote transfer (which, when it occurs, indicates meaningful learning) … retention requires that students remember what they have learned, whereas transfer requires students not only to remember but also to make sense of and be able to use what they have learned." Higher-order thinking is often defined in terms of critical thinking, e.g. as "reasonable, reflective thinking that is focused on deciding what to believe or do (Norris & Ennis, 1989, p. 3).There are actually multiple definitions of critical thinking.Ivie (2001, p. 10) defined critical thinking in terms of a reflective practice enabling learners to "establish clear and logical connections between beginning premises, relevant facts, and warranted conclusions."More recently, critical thinking has been characterized in terms of the metacognitive skill of evaluating one's thought process as self-guided and selfdisciplined (Paul, 1988).
Bloom's taxonomy (Bloom, Engelhart, Furst, Hill, & Krathwohl, 1956) provides a different approach to examining higher-order thinking skills.Geertsen (2003), who considers critical thinking and reflective thinking as distinct but complementary forms of higher-order thinking, proposes ten micro-thinking skills by degree of abstraction, as a revision of Beyer's (1987) model of Bloom's taxonomy.His top five micro-thinking skills by degree of abstraction, beginning with the most abstract, are (Geertsen, 2003, p. 3):


Extrapolation: to discover how what applies to known situations/data might apply to less familiar situations/data.


Evaluation: to determine the value of something  Explanation: to deduce/show how something logically follows  Synthesis: to put together to form some coherent whole  Analysis: to break down into various parts (compare and contrast) Argumentation and creative thinking are two additional higher-order thinking skills mentioned in the literature.Kurfiss (1988) points out that argumentation is a focal point in critical thinking, and other authors agree with him in considering it a higherorder thinking skill (Barak & Dori, 2009;Geertsen, 2003), and some even define critical thinking as argumentation (Yeh, 2001).King et al. (1998, p. 32) state that higher-order thinking skills, including creative thinking, "are activated when students of any age encounter unfamiliar problems, uncertainties, questions or dilemmas", and some authors define critical thinking as part of the process of evaluating the evidence collected in problem solving or the results produced by thinking creatively (Crowl, Kaminsky, & Podell, 1997;Lewis & Smith, 1993).
The literature includes other skills as higher-order thinking skills (e.g.problem solving, logical thinking, reasoning), but for our purposes those described above provide a broad perspective of the skills involved in higher-order thinking for the purpose of our paper, even as defining problem solving, critical thinking, reflective thinking, reasoning, and higher-order thinking continues to be a challenge (King et al., 1998).

Teaching higher-order thinking skills
A number of researchers emphasize the importance of students in all academic domains acquiring higher-order thinking skills (Levine & Ornstein, 1993;Newmann, 1988;Pogrow, 1988;1996;White & Frederiksen, 1998) and the literature is abundant with cases of teaching to promote higher-order thinking skills (e.g., Barak et al., 2007).Behar-Horenstein and Niu (2011) provide an overview of 42 different studies on measuring teaching critical thinking skills in higher education.They came to the general conclusion that most of the studies had many limitations and it is very difficult to prove a valid causal relationship between teaching intervention and development of higher order thinking skills.They also pointed out that to develop students' critical thinking skills, it is important to change the teaching paradigm from "what to think" to "how to think".As educators have been directed into teaching higher-order thinking skills, it has become clearer that there are difficulties not only in adapting the learning environment to teach these skills, but also in the assessment of the skills themselves.In a longitudinal study Miri et al. (2007) report that only two teachers from ten applied teaching strategies for promoting higher order thinking skills.Newmann (1988, p. 63) mentions that development of higher-order thinking skills might also be hindered because it "involves hard mental work".
A number of authors report on evaluating the use of concept mapping on learning or exercising higher-order thinking skills (Bixler, Brown, Way, Ledford, & Mahan, 2015;Chen, Liang, Lee, & Liao, 2011;Dabbagh, 2001;Ghani, Yahaya, Ibrahim, Hasan, & Surif, 2017;Green, 2010;Khodadady & Ghanizadeh, 2011;Rosen & Tager, 2014;Wheeler & Collins, 2003).However, this is a small number compared to the number of papers published on the effect of concept mapping on improved learning, in particular meaningful learning, which tends to be the main focus of the concept mapping community (see, for example, the proceedings of the International Conference on Concept Mapping (All proceedings are available at http://cmc.ihmc.us/cmc-proceedings/).We propose that researchers, educators, and concept mapping users in general -Cmappers, as the community likes to call itself (Cañas & Novak, 2008)associate concept mapping with higher-order thinking skills based on the experience they've had while constructing their own concept maps.Many years of leading concept mapping workshops for educators in many countries, from kindergarten teachers to senior research professors, have taught us that through the experience of learning how to construct concept maps, and the process of improving on their map construction both in terms of skills and quality of the map, the participants identify and openly express that they are exercising higher-order thinking skills and comment that they would like their students to experience this same process as they construct their maps.As a participant of a workshop expressed at the end of a long concept mapping day, "I'm tired of thinking so much".
The process of constructing a concept map is arguably more important than the final map, particularly if we are interested in learning and exercising higher-order thinking skills.When asked to construct a concept map, students are provided with instructions that include conditions (or a complete lack of conditions) for the process.The pre-given conditions provide varying degrees of freedom in terms of map content and structure (Cañas, Novak, & Reiska, 2012), as shown in Fig. 1, where the x-axis shows increased freedom of structure and the y-axis increased freedom of content.At the lower left corner of the figure, students are asked to memorize a concept map, a task that involves rote learning and thereby requires no higher-order thinking skills.At the top right corner of the figure the student is given a blank canvas and has complete freedom in terms of the content, including the topic of the map, concepts, linking phrases, and freedom on the structure of the map.Given that the student was not given any conditions, and since every concept map should respond to a focus question that provides the reference or context for the map (Novak & Cañas, 2008), the student must begin by preparing a question.Elaborating a good focus question is not easy.The focus question should guide the construction of the map, and during the process of construction other questions will arise, from the student, other students, or the instructor, that will lead to expansion of the map or construction of other maps.The process of construction should be a continuous questioning between the student and the educator, a higher-order thinking skill.
When the instructor provides the focus question, it somewhat restricts the student in both the content and structure of the concept map (see Fig. 1).It would seem that a focus question would restrict only the content of the map, but research has shown that a focus question may determine not only the content but also the structure of the map, and a dynamic focus question may lead to more explanatory concept maps (Derbentseva, Safayeni, & Cañas, 2006a) and cyclic concept maps (Safayeni, Derbentseva, & Cañas, 2005).A dynamic focus question usually provokes concepts being events instead of objects, which leads to dynamic propositions, resulting in explanatory instead of descriptive concept maps (Cañas & Novak, 2006).Providing the student with the root concept of the concept map as a starting point, which further restricts the content and structure of the resulting map (see Fig. 1), is not a common condition even though research has shown that providing a quantified root concept has a stronger effect on the resulting concept map than providing a corresponding focus.Furthermore, providing both a dynamic focus question and a quantified root concept leads to more dynamic thinking and results in explanatory concept maps (Derbentseva, Safayeni, & Cañas, 2006b), requiring that the student provide explanations, a higher-order thinking skill, instead of descriptions.The construction of the concept map consists of placing the concepts on the canvas and establishing the connections and linking phrases in a way that it expresses the map builder's understanding of a topic within the context provided by the focus question.The instructor could provide the student with a list of concepts (see Fig. 1), asking the student to either (a) use only the given concepts in the map, (b) use the given concepts and optionally add more concepts that the student feels are relevant, (c) use the given concepts in the map only if they are relevant.Each alternative will result in a different map and which to use depends on the objectives the instructor has when asking the students to build the map (Cañas et al., 2012;Kinchin, 2014).Providing or not the concepts to the student has an effect in the resulting map, but does not affect the higherorder thinking skills involved in the construction of the map.The same is the case if a scaffolding, skeleton concept map (see Fig. 1) is provided as a starting point for the student (Novak & Cañas, 2008).It's placing the concepts on the canvas, and the construction of the propositions, and specifically the selection of the linking phrases during that construction of the propositions, that invokes higher-order thinking skills.In the construction of a hierarchical concept map, which is the most common topology, the student must determine which are the more general conceptsthe big ideas -and place them on the upper part of the map, and then place the more specific concepts at the bottom of the map (this hierarchy of concepts is in accordance with Ausubel's Assimilation Theory on which concept maps are theoretically based (Ausubel, 1968;Novak & Gowin, 1984)).
The organization of the concepts and construction of the propositions is an iterative process where the concepts are moved around, modified, added and deleted and the linking phrases are added and refined.The map constructor at this point is constantly analyzinga higher-order thinking skillwhether the latest (or next) actions (move the nodes around, add a linking phrase, etc.) best represents his or her understanding of the topic.Time is often spent on thinking which of several possible linking phrases best reflects the relationship between two concepts.The linking phrases are often replaced by a better alternative, concepts may be added, deleted, or moved around, and the big ideasthe most relevant concepts of the mapmay become evident as the map is constructed, if they haven't already.This construction and refinement process is one of reflective thinking, a higher-order thinking skill.The student, possibly through the interaction with the instructor is constantly reminded to ask him or herself, "does the concept map answer the focus question?", a process that requires not only reflective thinking but also evaluation, a higher-order thinking skill.
Crosslinks, relationships or links between concepts in different segments or domains of the concept map, show how a concept in one domain of knowledge represented in the map is related to a concept in another domain in the map.In the creation of new knowledge, crosslinks often represent creative leaps on the part of the knowledge producer and are considered important in the facilitation of creative thinking (Novak & Cañas, 2008), a higher-order thinking skill.The presence of crosslinks in the map is key for the student to show a broad understanding of the topic, and is usually encouraged through the interaction with, and questioning from, the instructor.
The construction of a concept map is seldom a one-time task (sit down and complete the map) if the concept map is to be used as a learning tool.If the student is asked to build a map as part of an assessment exercise, without any possibility of refining the map, feedback or questioning from the instructor or peers, or the chance to learn about the topic to improve and refine the map, there is little opportunity for higher-order thinking.If the concept map construction is part of the learning process, and building the map is part of the challenge of learning, then the higher-order thinking skills are activated.We propose that concept mapping be used during the whole instruction of a unit (or course) to reflect the advancement in student's learning across time (Novak & Cañas, 2004).Starting with a concept map at the beginning of the unit that shows the student's previous knowledge on the topic given a list of concepts, or beginning with a skeleton map, or with a blank canvas, but always with a good focus question, the concept map is periodically updated to reflect what the student has learned as he or she has advanced through the unit.In this iterative process, the concept map becomes a means by which the student interprets and represents the acquired knowledge, showing the increased understanding.Questioning is continuous by the student, peers and the instructor, leading to further learning and expansion of the concept map.The concept map becomes a means by which the student gets to determine what is still to learn (if I can't map it, I don't understand it), a metacognitive processand a higher-order thinking skill.We understand that this requires a change in the learning environment, but as we discussed earlier, that's the case when trying to teach any higher-order thinking skill.Cañas, Novak, and Reiska (2015) distinguish between good and excellent concept maps, as depicted in Fig. 2. A good concept map, referred to as Level 2 or L2, has a good structure and good content.An excellent concept map (referred to as Level 3 or L3) not only responds to the focus question, it explains the in a clear fashion.It is concise, all concepts and propositions are relevant to the topic of the concept map, thus does not include "unnecessary" concepts, propositions or crosslinks, and is not missing any key, relevant concepts, propositions or crosslinks.An excellent concept map has high clarity, a clear message, and communicates key ideas.Excellent maps are explanatory, not descriptive.Constructing excellent concept maps requires not only all the higherorder thinking skills mentioned in this section (questioning, explaining, analyzing, reflective thinking, interpreting, metacognition, etc.), but in addition requires a high level of synthesisa higher-order thinking skillto be able to express knowledge in the most concise and explanatory form.Our research has shown that there is a correlation between the quality of concept maps constructed by students and their level of scientific literacy (Reiska, Soika, Möllits, Rannikmäe, & Soobard, 2015), and the literature has reported a correlation between levels of scientific literacy and higher-order thinking skills (Deming, O'Donnell, & Malone, 2012).But, it takes time and practice to become an Excellent Cmapper, and the degree of higher-order thinking that is involved increases as the maps approach L3.
We have used the singular 'student' throughout the discussion on concept mapping, but frequently students perform the construction of concept maps in groups.
While constructing a concept map as a group, students must carry on a negotiation of meaning, in particular during the construction of propositions.This meaning negotiation, in fact, is one of the richest learning aspect of concept mapping, and leads to a constant argumentation, a higher-order thinking skill, among students and between the student(s) and the instructor.Argumentation and discussions are further enhanced if the concept mapping process includes complementary tools such as the Knowledge Soup (Cañas et al., 2001), where students publish and share propositions (referred to as 'claims' in the Knowledge Soup) and can question and discuss each other's claims.
The process of constructing a concept map is rich with higher-order thinking skills.However, not all students benefit from the opportunity to live concept mapping as we've described above, largely because of frailty on the part of their instructors as we'll describe in the next section.

Reaching concept mapping skills that require higher-order skills
Concepts maps are deceivably simple: an excellent concept map consists of a set of nodes with text in them, that are linked through linking phrases in such a way that it can be read and understood without having somebody explain it to you.If you read it, it just makes sense.We like to compare an excellent Cmap to a poem: "Anyone can write lines of verse, but that doesn't make a poem" (Cañas et al., 2015, p. 14).However, getting to build good concept maps is hard, and excellent concept maps much harder.There is a learning curve in constructing concept maps that is usually reflected in the concept map's size, as shown in Fig. 3.The Figure depicts the time from when users starts concept mapping to the time they reach Excellent Cmapper.New users, learning the mechanics of putting together a concept map, begin with small maps, more so if they are learning to use a software tool such as CmapTools (Cañas et al., 2004) at the same time.As they become more comfortable with concept mapping, their maps become larger and now the concern is not so much with the mechanics but with the content (What concepts am I missing?Is my concept map large enough?).At this stage they are usually struggling with finding the appropriate linking phrases but not yet spend a lot of time thinking about the best linking phrase to construct a proposition.Users who continue to spend time concept mapping, who get feedback on their maps, and use concept mapping as a learning tool, start spending more time on the aspects that lead to an excellent concept map: are the main, key concepts (and ideas) clearly represented?Are all concepts and propositions relevant to the topic of the map and answer the focus question?Are all relevant concepts included?Are propositions constructed with the best, most appropriate linking phrase?Could the concept map be more concise?Is the focus question the right question?Does the concept map respond the focus question?etc.The Cmapper is going through the point in time depicted by the dotted line in Fig. 3.It is to the right of the dotted line that concept mapping requires the most of the higher-order thinking skills described above.It's not the size of the concept map that is important any more, but what the concept map explains and how it explains it.In concept mapping, larger is not always better (Kinchin, 2014).So, yes, concept mapping is easy to learn and use (Fiedler & Salas, 2004;Wei & Yue, 2016), but this simple construction leaves you at the left of the dotted line in Fig. 3.
If we expect students to benefit from concept mapping with regard to learning and exercising higher-order thinking skills, they must reach the point in terms of their concept mapping skills to the right of the dotted line in Fig. 3.However, few students ever reach that point.Unfortunately, most students stay somewhere to the left of the tipping point of the curve.

Concept mapping, higher order skills, and pedagogical frailty
There are many reasons why most students never reach beyond the tipping point of Fig. 3. To begin with, we believe that students are seldom given the time to develop their concept mapping skills.In most cases the student's first attempt at constructing a concept map is usually already part of an assignment, after maybe some time to learn to use a concept mapping software and a few guidelines from the instructor on how to build maps, e.g.videos or links to websites.Again, there are a variety of reasons why this is the case.Often instructors state that they don't have time to spend teaching students how to construct concept maps since they have to cover the course syllabus.This of course is the chicken-and-egg problem: if students learned how to be Excellent Cmappers in their first interaction with concept mapping, instructors and courses later in their academic program would not have to deal with teaching concept mapping.As a result, students don't have an opportunity to build their concept mapping skills before asked to build concept maps as part of evaluated assignments.
Just as we have argued that it takes time and effort to become an Excellent Cmapper, it's also true that constructing your first concept map is very easy.Bixler et al. (2015) report that when provided concept mapping in a modest dose, students were not able to improve critical thinking skills, but students perceived the process of concept mapping as easy to use and learn.Soika and Reiska (2013) report that in a project that involved asking 1614 Estonian High School students to construct a concept map as part of an experiment, many of the students had no experience constructing maps while the rest had some, but none had extensive experience and had not reached the right side of the curve.This was reflected in the results where there was no correlation between the quality of the concept map and the mapping experience.Students with no experience were able to do just as well building the maps as students with some experience.But we are aware that in general these students did not fully exercise their higher-order thinking skills during the construction, and the purpose of the mapping was assessment, which we propose is one of the reasons why advanced concept mapping skills are not reached.
Since the most common use of concept mappingand in fact the use that provoked the development of the concept mapping tool (Novak & Cañas, 2006) is instructors' assessment of students' understanding, students are usually given one shot at getting the concept map 'right' with the instructions to draw the map and hand it in as part of an assignment or examination.Given the tendency of concept map assessment rubrics to rely on structural measurements (Kinchin, 2014), students will add as many concepts and propositions as possible, a large hierarchy, and lots of crosslinks and branching points, to make sure they score well on the rubric (students are particular adept at figuring out their instructor's assessment rubrics).As we stated earlier, in concept mapping, larger is not always better: Excellent Cmappers can select the key concepts and the linking phrases that best provide an explanation to the given question, resulting usually in smaller maps (Cañas et al., 2016;Kinchin, 2014).
The conditions under which the concept mapping exercises described above are carried out don't provide students much opportunity to learn or develop higher-order thinking skills.Students are seldom given a good focus question, there is no feedback or questioning by part of the instructor or peers on the maps, there are few or no opportunities to revise the maps, and, as stated above, instructors will rely on rubrics that favor quantitative scoring as they don't have time to read and try to understand each map.The construction of the concept map is not part of the learning process, it's part of the assessment.
Students who go through several concept mapping exercises of this type often end up close to the peak of the curve, on the left side.They can build large concept maps, with reasonably well constructed propositions, such that they are structurally and contentwise 'good' (L2, 'good' concept maps, Fig. 2) (Cañas et al., 2015).As can be inferred from the description above of the concept mapping process that lead to learning and exercising higher-order thinking skills, these students are missing a large part of this process in their mapping experience.
On the other hand, students who are given the opportunity to work on their maps as they learn the subject matter and get feedback from the instructors and peers will begin to apply their higher-order thinking skills, resulting, for example in discriminating between key and irrelevant concepts, their maps becoming better built structurally and content-wise, and more precise, as was reported by Dowd, Duncan, and Reynolds (2015).
What went wrong?Instructors seem to be generally aware that concept mapping leads to the development and exercise of higher-order thinking skills, both from the literature (e.g., Bramwell-Lalor & Rainford, 2014;Hill, 2004;Won, Krabbe, Ley, Treagust, & Fischer, 2017) and based on our interactions with them.We stated earlier that to develop students' critical thinking skills it is important to change the teaching paradigm from "what to think" to "how to think", and in support of the development of these skills, we need to change towards using concept mapping to learn how to think.But using the concept map as a learning tool requires changes in the learning environment and teaching strategies (Kinchin, 2001), in the role of the instructor and the student, where students are more active in their own learning, and just as we reported above regarding teaching higher-order thinking skills, it is becoming clearer that there are difficulties in adapting the learning environment to teach these skills.
We propose that the lack of development of higher-order thinking skills while using concept mapping in the classroom as explained above is a case of pedagogical frailty.Society as a whole understands the importance and promotes the teaching of higher-order thinking skills at all educational levels.Academic and educational institution leaders and instructors seem to be in principle aligned with this understanding.So, there doesn't seem to be a lack of explicit and shared values, the 'Regulative Discourse' dimension proposed to contribute to pedagogic frailty by Kinchin (Kinchin et al., 2016;Kinchin & Winstone, 2017).However, there seems to be a disconnection between the practices of concept mapping in the classroom with the pedagogy that underpins the teaching of concept mapping, the 'Pedagogy and Discipline' dimension proposed to contribute to pedagogic frailty (Ibid).This is shown in the pedagogic frailty model in Fig. 4.This disconnection could be due to the instructor not really understanding how to fully take advantage of concept mapping as we have described.In other words, the instructor believes that concept mapping does help develop and practice higher-order thinking skills but doesn't know how to accomplish it.We believe that a reason for this could be that most instructors never get to be Excellent Cmappers, never reached beyond the dotted line of the curve in Fig. 3.If the instructors have not personally experienced this advanced use of concept mapping, it can't be expected that they take full advantage of the tool.There is little research on the level of concept mapping of instructors.Miller, Cañas, and Novak (2008) report that elementary school teachers did not advance to what we interpret would be the right side of side of the dotted line by the end of a concept mapping workshop, and in their study Subramaniam and Esprívalo Harrell (2015) report that future elementary and/or secondary classroom teachers did not exhibit the necessary knowledge for managing concept mapping in their classrooms.There is anyway a disconnect between the practices of concept mapping and the pedagogy behind it, a case of the 'Pedagogy and Discipline' dimension of pedagogic frailty.But it could also be that instructors fully understand that students are not reaching the level of concept mapping that would develop higher-order thinking skills, a case analogous to what Barak et al. (2007) report as teachers not using all possibilities to develop critical thinking skills of high school students.This would clearly be a case of pedagogic frailty.There may also be a disconnection between the instructors and the decisionmaking bodies that regulate teaching, the 'Locus of Control' dimension of pedagogic frailty (Kinchin et al., 2016;Kinchin & Winstone, 2017) (see Fig. 4).We mentioned that the learning environment needs to change in order to promote the development of higherorder thinking skills, and in particular when doing so through concept mapping.There may be cases where the academic authorities do not allow, directly or indirectly, those changes to occur.For example, a high school curriculum that is imposed on an instructor together with how to deliver it, together with the need to deal with standardized testing may limit the changes the instructor is able to implement.The authorities may in principle promote the learning and development of higher-order thinking skills but in practice impede its implementation through other regulations and requirements.The impedance may even be furtive, where fellow instructors, principals and/or supervisors discourage any changes that would badly show on them, who don't want to deal with changes.

Conclusions
Even though concept mapping can be an effective tool in the development of higher order thinking skills in students, and it's practically a universal objective within the educational organizations to promote these skills, we find that in many cases where concept mapping is used, students don't seem to reach the level of expertise and experience in concept mapping needed to take full advantage of the develop and exercise these skills.We discussed in this paper that the reasons for not reaching the concept mapping skills required can usually be traced back to the instructors, the way they use concept mapping with their students, their understanding of concept mapping, and the learning environment and methodology they practice.These issues can be understood within the framework of pedagogic frailty, and specifically within the 'Pedagogy and Discipline' and 'Locus of Control' dimensions as shown in Fig. 4.
We have observed that although there is substantial literature on the effects of concept mapping in improving learning, in particular meaningful learning, there is little research on the use and/or effect of concept mapping in developing higher-order thinking skills, even though the literature frequently refers to concept mapping as an effective tool to develop these skills, in particular critical-thinking.
Finally, there is also little research reported in the literature on the level of skills and understanding of concept mapping held by instructors using concept mapping with their students.This may be one good starting point for further research on the development of higher-order thinking skills in students through the use of concept mapping.

Fig. 1 .
Fig. 1.Freedom of structure and freedom of content conditions during concept mapping, from Cañas et al. (2012)

Fig. 4 .
Fig. 4. The two dimensions of the Pedagogic Frailty model that affect the development and exercising of higher-order thinking skills when concept mapping, adapted from Kinchin et al. (2016)