How Do Technology-Enhanced Learning Tools Support Critical Thinking?

Le, Nguyen-Thinh

doi:10.3389/feduc.2019.00126

REVIEW article

Front. Educ., 06 November 2019

Sec. Digital Education

Volume 4 - 2019 | https://doi.org/10.3389/feduc.2019.00126

How Do Technology-Enhanced Learning Tools Support Critical Thinking?

$\nNguyen-Thinh Le$ Nguyen-Thinh Le^*

Computer Science Education, Computer Science and Society, Department of Computer Science, Humboldt-Universität zu Berlin, Berlin, Germany

This paper reviews existing computer-supported learning systems that have claimed to adopt Socratic methods for enhancing critical thinking. Several notions of Socratic methods are differentiated: the critical thinking framework of Paul and Elder (2006), the classic Socratic method, the modern Socratic method, and the neo-Socratic group discussion method. Three lessons are highlighted. First, the development of learning systems specifically supporting critical thinking is still lacking Thus, further research in this area is urgent. Second, most developed computer-supported learning systems claim to support Socratic approaches (e.g., Socratic tutoring) which are based on human tutoring strategies and do not show a systematic Socratic method. Third, the classic Socratic method has not been applied in any reviewed learning system.

Introduction

What is critical thinking? The definition of Sumner (1940, p. 632–633) might be one of the earliest notions of “critical thinking”: [Critical thinking is] “…the examination and test of propositions of any kind which are offered for acceptance, in order to find out whether they correspond to reality or not.” This notion implies active scrutiny of propositions when articulated. Similarly, most definitions share the common requirement on question asking. That is, the critical thinker needs to ask questions in order to test assumptions, to recognize ambiguity, to examine, to interpret, to evaluate, to reason, to reflect, to clarify, to articulate, and to justify positions (Ennis, 1962; Ruggiero, 1975; Hallet, 1984; Halpern, 1996). However, none of these definitions provides a systematic framework for adoption in educational scenarios.

In 2012, Richard Paul published an article criticizing the education of critical thinking at schools as follows: “The fundamental problems in schooling today at all levels are fragmentation and lower order learning. Both within and between subject areas there is a dearth of connection and depth. Atomized lists dominate curricula, atomized teaching dominates instruction, and atomized recall dominates learning. What is learned are superficial fragments, typically soon forgotten. What is missing is coherence, connection, and depth of understanding…” (Paul, 2012). Many empirical studies reported a similar situation of critical thinking education at schools. Most teachers and school students do not use deep questions that are supposed to evoke high-order cognitive functions (Graesser et al., 2010; Chafi and Elkhouzai, 2014). Thus, students have limited exposure to more beneficial inquiry. Approximately 60% of teachers' questions evoke lower-order cognitive demands, whereas 20% invoke higher-order cognitive demands, leaving 20% that represent procedural day-to-day questions (Dickman, 2009). A recent study conducted with 143 teachers in Germany expressed a similar result that low-order questions are mostly used in classroom teaching (Le et al., 2018).

Critical thinking is the skill that is in high demand in many workplaces nowadays. For global industry groups such as the World Economic Forum, critical thinking has been consistently ranked as one of the top three most important skills from 2015 to 2020 (WEF, 2016). Despite the importance of critical thinking in education, research on technology-enhanced support for developing and enhancing critical thinking is still rare. The goal of this paper is to investigate the research question: How do existing technology-enhanced learning tools help learners develop critical thinking? Answering this question should also shed light on associated pedagogical practices. As a first step, the discussion focuses on the Socratic methods and its relationship with critical thinking.

Methodology

In order to investigate the research question being addressed in this paper, first, it is required to review different approaches to develop critical thinking in order to be able to classify learning tools. Thus, the following sections are devoted to differentiating variants of Socratic approaches to critical thinking.

The Paul-Elder's Socratic Approach to Critical Thinking

One of the pioneers of promoting critical thinking in education is Richard Paul. Paul's definition for critical thinking is as follows: “Critical thinking is disciplined, self-directed thinking which exemplifies the perfections of thinking appropriate to a particular mode or domain of thought.” Paul suggested the following twelve criteria for perfections of thought: clarity, precision, specificity, accuracy, relevance, consistency, logicalness, depth, completeness, significance, fairness, and adequacy (for purpose). These criteria for perfections of thought can be used to assess the level of critical thinking, and thus, are also referred to as the intellectual standards (Paul and Elder, 2006). In order to achieve the perfections of thought, Paul suggested six categories of questions for critical questioners (Paul, 1990, Chapter 19) (see Table 1).

TABLE 1

Table 1. Six classes of critical questions proposed by Paul and Elder (2006).

By applying the six classes of critical questions, the development of social intellectual traits might be expected (Paul and Elder, 2006). The criteria for intellectual standards of critical thinking and the six categories of questions build a framework of critical thinking.

The Classic Socratic Method

The classic Socratic method originated primarily from the early dialogues of Socrates that are documented in Plato's books (Maxwell, 2014). In these dialogues, Socrates used questions to refute existing beliefs of the interlocutor. Such refutation allows the interlocutor to rethink the topic under discussion (e.g., “What is virtue?”). The expected result of the classic Socratic method is that the interlocutor can recognize by himself/herself the failure during the process of searching for a correct answer to a discussion question. Another expected effect is that the interlocutor would rethink his/her existing belief more deeply and free himself/herself from holding firmly to his/her wrong belief. This is referred to as the “Socratic effect” by Maxwell and Melete (2014). Through this effect, new knowledge of the interlocutor may be established.

Boghossian (2012) identified five common steps of the classic Socratic method: (1) Wonder question, (2) Hypothesis, (3) Elenchus (refutation or cross-examination), (4) Acceptance/rejection of the hypothesis, and (5) Action. The first step starts with a wondering question, e.g., “What is justice?” (Chapter “The republic,” Plato¹). The second step of a Socratic dialogue is the response of the interlocutor who is in charge by presenting a hypothesis, a possible answer or a tentative answer to the question. In this stage, the interlocutor may use his/her knowledge to answer the “wonder” question asked by Socrates. The answer shows the pre-conception of the interlocutor and represents a hypothesis. Socrates would not evaluate the answer given in this stage. The third step of a Socratic dialogue, elenchus or refutation, is the core of Socratic dialogues (Gulley, 1968). The purpose of this step is to ask questions to test the hypothesis given by the interlocutor. The hypothesis could be tested by elenchus (refutation or cross-examination, e.g., fact check, critical questions, counter-arguments, counter-examples, fallacy-check, or check for contradiction, etc.). The purpose of the elenchus (refutation or cross-examination) is to call the hypothesis into question. That is to undermine the interlocutor's belief. The fourth step of a Socratic dialogue is to accept or reject the hypothesis of the interlocutor based on results of rethinking. If a new fact (or counterexample, counter-arguments, fallacy-check, check for contradiction) shows that the hypothesis cannot be true, then the interlocutor should change his/her belief. He/she goes back to the second step and offers another hypothesis. If a new fact (or counter-arguments, fallacy-check, check for contradiction) is rejected by the interlocutor, then both the Socratic questioner and the interlocutor agree that it is neither necessary nor sufficient to undermine the hypothesis. That means that the hypothesis is tentatively accepted. The final step is to act by the interlocutor accordingly, after the cycle of examining facts (or counterexamples, counter-arguments, fallacy-check, check for contradiction) has been finished. That is, one would change his/her pre-conception.

Maxwell and Melete (2014) compared the five steps of the classic Socratic method with the general steps of the scientific approach to investigating a research question. An example from Meno (Jowett, 2019) illustrates the classic Socratic method as follows, sentences in italics are my notes indicating the steps of the classic Socratic method.

The classic Socratic method has been proven useful in teaching and learning (Lam, 2011). However, several researchers argued that the classic Socratic method tends to confuse and to perplex students (Pekarsky, 1994; Tarnopolsky, 2001; Weisner and Westerhof-Shultz, 2004) and that students may become humiliated and ashamed. Boghossian (2012) represented the opposite point of view by showing different examples: “The purpose of the Socratic method is not to humiliate, shame, or perplex students, but to help them have beliefs that accord with reality.” For Boghossian, the classic Socratic method has much potential: it can help participants formulate arguments, improve their critical thinking and moral reasoning skills, and learn to distinguish truth from falsity. The perplexed and confused feelings are just the side-effect of the classic Socratic method (Boghossian, 2010). Socratic dialogues, as described above, aim only to free one's wrong belief from holding tightly on to previous convictions, and thus evelop critical thinking.

The Modern Socratic Method

Maxwell (2014) distinguished the modern Socratic method from the classic Socratic method. The modern Socratic method uses questions to lead the interlocutor to acquire knowledge in small steps. This means that the answers of leading questions can be verified and anticipated by the Socratic questioner. This is the main difference between the modern Socratic method and the classic Socratic method such that neither the Socratic questioner nor the interlocutor knows the answer. According to Maxwell, this Socratic method is popular in modern times and thus, referred to as the modern Socratic method. This type of Socratic method is also the root of the dialogues of Socrates. One of the Socrates' dialogues that can illustrate this method is the conversation between Socrates with a slave boy about the geometry experiment found in the dialogues “Meno” (Meno 82b−85d: Socrates and the Slave²). A part of this dialogue is shown in Figure 1.

FIGURE 1

Figure 1. An illustration of modern Socratic dialogues (Meno 82b−85d: Socrates and the Slave, Source: Wikimedia.org).

The Neo-Socratic Discussion Method

Nelson (1970) developed a Socratic discussion method which is referred to as the neo-Socratic method in literature (Popp, 2001). This method is intended to support a group discussion for six to ten participants. The discussion serves to explain existing but unreflected concepts in daily life (e.g., What is happiness?) that are fundamental for the discussion. Through a discussion held by the neo-Socratic method, the participants perform argumentation and strive for a result in consensus. Similar to Socratic dialogues that can be found in the books of Plato, the neo-Socratic discussion method applies concrete examples in daily life for self-reflection. Based on self-experience, the participants express their points of view on the discussion question. The central point of this method is the enhancement of self-initiated thinking, the improvement of the ability of logical and objective argumentation, and the promotion of problem-oriented and solution-oriented communication. Heckmann (1981) extended Nelson's neo-Socratic method by explicitly defining the rules for the discussion moderator and for discussion participants. With these rules, Heckmann (1981) wanted to make sure that the abstraction process from examples given by discussion participants is granted. Horster (1994) investigated the theoretical assumptions of the neo-Socratic method, modified the abstraction process proposed by Nelson, and described the neo-Socratic method as Figure 2 illustrates. The steps of this process are elaborated by Horster (1994). Since this abstraction process of the neo-Socratic method seems to be clearly defined, it could be mapped to a computational model.

FIGURE 2

Figure 2. The Socratic group discussion method developed by Nelson (1970), extended by Heckmann (1981) and Horster (1994).

The differences between the classic, the modern, and the neo-Socratic discussion methods are summarized in Table 2.

TABLE 2

Table 2. The differences between the Socratic methods.

Reasoning

Socratic questioning not only involves the use of systematic questioning, but also inductive reasoning (Carey and Mullan, 2004). Inductive reasoning uses specific examples to arrive at a general rule. For example, we can observe from specific examples that a bicycle has two round wheels, a motor bike also has two round wheels, and a car has four round wheels. We would induce a general rule that all vehicles have round wheels.

Results

The foregoing investigation of Socratic Methods is presented as important context for understanding the application of contemporary technology support for critical thinking, for the main reason that most systems have adopted the modern Socratic Method. This discussion now addresses findings associated with this.

For the review of technology-enhanced learning systems for critical thinking, the following inclusion criteria were defined:

1. Scientific articles describing a technology-enhanced learning system must mention “critical thinking” or “Socratic” (including “Socratic dialogue,” “Socratic method,” “Socratic questioning”), and “reasoning”;

2. A system must have educational purposes, e.g., learning, developing/enhancing skills;

3. A system must have been evaluated or technically validated.

In addition to the inclusion criteria, one exclusion criterion is that assessment systems are not considered, because they do not provide didactic/pedagogical strategies to enhance critical thinking skills.

Applying these three inclusion criteria and the exclusion criterion, articles were collected from Google Scholar, DBLP and open access journal databases on the Internet, 14 learning systems for critical thinking were included (Table 2). In the following, each system is briefly summarized and assigned to one of the critical thinking approaches. If the authors of the system claimed that it supports the Socratic method but did not show the systematic Socratic method, we will assign that system to the category “claimed to be Socratic.” If a system is still available online, it is indicated by an Internet URL on a column of the table.

The review starts with the learning systems that adopt the modern Socratic method. The common feature is that the systems control the dialogue, ask questions and the students answer the system's questions in free text. One of the earliest computer systems that adopted the modern Socratic method is SCHOLAR (Carbonell, 1970). In this system, the author modeled the domain geography using a semantic network. The system allows mixed initiative dialogues, i.e., both students and the system can initiate questions. The user interface allows the users to input an answer or a question in free form. The system understands the student's question or answer by matching a pattern with pre-specified keywords. In order to generate texts, the system fills answer and question templates with information from the semantic net. Since the semantic network represents only fact knowledge rather than procedural knowledge, the system is limited to categorize student utterances beyond simply right/wrong. Also adopting the modern Socratic method, Weusijana et al. (2004) developed a questioning strategy for the system SASK. It is a domain-independent architecture for deepening students' reflections on well-defined tasks using Socratic dialogues. In the domain of biomedical engineering, for example, the system adopts the questions used by experts for students such as “What are you trying to do here?” or “What variables are you controlling?” Person and Graesser (2002) developed an intelligent tutoring system that applies the modern Socratic method to improve students' knowledge in the areas of computer literacy and Newtonian physics using an animated agent that is able to ask a series of deep reasoning questions according to the question taxonomy proposed by Graesser and Person (1994).

Beside the learning systems that applied the modern Socratic method, several learning systems adopted the neo-Socratic group discussion method. Le and Huse (2016) developed a conversational agent that plays the role of a moderator for a group discussion. The conversational agent leads the discussion participants through the phases of the neo-Socratic group discussion method and encourages participants to strengthen their critical thinking in order to develop arguments for the given discussion topic. The evaluation study of the Socratic conversational agent (Le and Huse, 2016) reported encouraging results that the Socratic group discussion moderated by a conversational agent has the tendency to activate participants' thinking and join the group discussion more actively. For similar purpose, Hoeksema (2004) developed a group discussion environment that is intended to serve virtual Socratic dialogues. The Socratic dialogues using this discussion environment are intended to be held similarly in a usual face-to-face environment. Whereas, this work focused on developing an environment for Socratic group discussions, the Socratic conversational agent of Le and Huse (2016) was used to formalize the neo-Socratic group discussion method to help students develop critical thinking.

While the classic and modern Socratic methods are based on the dialogues of Socrates documented in the books of Plato, the conceptualization of the Socratic method has been developed and modified in different guises.

Edelson (1996) developed a so-called Socratic case-based architecture Crimeanate using thought-provoking questions and cases. Two pedagogical principles underlying this architecture are active learning and learning from cases. These principles are implemented by two system components: a task environment and a storyteller. The learning domain supported by this architecture is biology. Specific subject matter is animal adaptation. A session begins with an invitation to the student to create his or her own animal by taking an existing animal and changing it in some way. Following the choice of an animal, the system engages the student in a series of natural language dialogues in which the student considers the ramifications of the proposed modification of his or her animal. The storyteller recognizes opportunities for learning during the course of interactions of the student with the task environment and presents cases that may help the student to learn from his/her own problem.

Glass (2001) developed CIRCSIM, a dialogue-based intelligent tutoring system that uses questions to lead conversations with student and claimed that the pedagogical strategy is Socratic tutoring. This tutoring strategy is based on a corpus of human tutoring dialogues that contains many instances of students' short answers (Glass, 2001). The notion of Socratic tutoring suggested by Glass is as follows: “The dialogue is under the tutor's control; the machine asks questions and the student answers with free text in imitation of the Socratic style of human tutoring.”

Similarly, Weusijana et al. (2004, p. 561) characterized a Socratic tutoring method very informally: “An educator may know of these issues and choose to tutor their learners socratically; to conversationally engage with learners, often while they work on their learning task, with pertinent and probing questions.” Based on this concept of the Socratic method, the authors developed a web-based system that helps students foster reflection.

Domeshek et al. (2002) conceptualized the Socratic method as follows: “Socratic instruction is a kind of teaching interaction typically applied in high-level professional education (e.g., law and business) and most often characterized by its external form: the teacher asks a lot of questions, and the student answers.” Based on this notion of the Socratic method, Domeshek et al. (2004) developed ComMentor, an automated Socratic tutoring system, for command skills for high-level professional education such as law and business. This system is claimed to be able to guide the student in a Socratic mode as an expert would: the teacher asks questions and the student answers. The sequence of the questions is intended to help the student reconstruct the logic of expert situation analysis and decision-making. Domeshek et al. (2004) described four characteristics of a typical Socratic session: (1) a thought-provoking problem, (2) a student's attempt to provide solutions, (3) the instructor's repeated exploration and challenging of the student's solutions, and (4) incremental justification, elaboration, refinement, and revision of both the student's understanding of the situation under discussion and their proposed solution.

According to the notions for the Socratic method above that are not based on the analysis of Socrates' dialogues, a teacher should engage students by posing questions. It is controversial whether these notions for the Socratic method can be categorized as the modern Socratic method because the modern Socratic method also applies a sequence of questions for that the Socratic questioner anticipates correct answers. However, since the computer applications that adopt these notions for the Socratic method are based on the analysis of human tutoring dialogues, it is questionable whether these dialogues follow a systematic methodology and whether the methodology of human tutors is really effective.

Several educational applications support tutorial dialogues. Olney et al. (2012) presented a method for generating questions for tutorial dialogue. This involves automatically extracting concept maps from textbooks in the domain of biology. Five question categories were deployed: hint, prompt, forced choice question, contextual verification question, and causal chain questions. Also, with the intention of supporting students using conversational dialogues, Lane and VanLehn (2005) developed PROPL, a tutor, which helps students build a natural-language style pseudo-code solution to a given problem. All these educational applications deployed some kinds of dialogue, however, they neither apply the classic nor modern Socratic method.

There have been several computer-supported learning systems for human reasoning which could be considered a part of the critical thinking process since critical thinking involves the use of inductive reasoning (Carey and Mullan, 2004). For example, the framework of critical thinking proposed by Paul and Elder (2006) includes the class of questions that probe reason and evidence. Le and Wartschinski (2018) proposed a cognitive assistant that holds conversation with students to develop human reasoning skills. This study, with more than 60 test persons, showed significant improvement in reasoning skills. Pursuing the similar aim, an existing serious game, Argotario (Habernal et al., 2017) addressed argumentation and critical thinking skills by identifying fallacies in arguments and intentionally developing fallacious arguments during the process of playing a game. Both the cognitive assistant developed by Le and Wartschinski (2018) and the serious game Argotario proposed a conversational agent as the communication interface with the user. However, the difference between these systems lies in the training tasks. The cognitive assistant developed by Le and Wartschinski (2018) covered several issues that lead to irrational thoughts and decisions: (1) sunk cost fallacy, (2) gambler's fallacy, (3) Bayesian reasoning, (4) belief bias in syllogistic reasoning, (5) regression toward the mean, (6) co-variation detection, and (7) Wason's selection tasks. Training tasks provided by this cognitive assistant were based on psychology literature (Larrick, 2004; Toplak et al., 2014). The serious game, Argotario, only addressed the single issue of “fallacy.”

From this review of technology-enhanced learning systems for critical thinking (Table 3), we can learn three lessons. First, the number of developed learning systems for critical thinking is still low. Thus, given the proliferation of misinformation and ‘fake news' on the web, further research in this area is arguably urgent. Second, most of the developed learning systems (e.g., Olney et al., 2012) claimed that they support Socratic approaches (e.g., Socratic tutoring), which are based on human tutoring strategies rather than Socrates' strategies. It is controversial whether the human tutoring strategies are pedagogically effective and whether they need to be empirically validated before being integrated into a learning system. Third, the classic Socratic method has not been applied in any reviewed learning system. This absence of the classic Socratic method in learning systems can be explained by which the steps of the classic Socratic method might be very challenging to be mapped to a computational model. Especially the third step, which is the core of the classic Socratic method, would require a computer system to be able to ask a question to test a hypothesis by posing a fact check, a counter argument, counter example, a fallacy check, or a check for contradiction.

TABLE 3

Table 3. A summary of computer-supported educational systems for critical thinking.

Conclusions

This paper has reviewed 14 existing technology-enhanced learning systems for critical thinking. The review shows that almost all existing systems adopted the notion of the modern Socratic method, e.g., the system uses questions to lead the learner to acquire knowledge in small steps and knowledge that is to be acquired can be anticipated by the system. Thus, questions and anticipated knowledge of a learning domain can be modeled computationally. Whereas, the modern Socratic method has been adopted in many systems, the classic Socratic method is rarely deployed in computer-supported learning systems. Perhaps the reason is that steps of the classic Socratic method are challenging to be mapped to a computational model. Another finding is that several dialogue-based learning systems claimed to adopt Socratic questioning method, however, they only support conversation between users and the system in natural language. That is, those systems may enhance critical thinking through questions, but a systematic Socratic approach cannot be identified.

Author Contributions

The author confirms being the sole contributor of this work and has approved it for publication.

Conflict of Interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

I acknowledge support by the German Research Foundation (DFG) and the Open Access Publication Fund of Humboldt-Universität zu Berlin.

Footnotes

1. ^Plato: “The republic” available on The Internet Encyclopedia of Philosophy (IEP) https://www.iep.utm.edu/republic/

2. ^Meno 82b - 85d: Socrates and the Slave https://upload.wikimedia.org/wikipedia/commons/d/d3/Meno82b-85d.svg

References

Boghossian, P. (2010). Socratic Pedagogy, Critical Thinking, and Of-fender Programming. Offender Programs Report.