Exploring the impact of Stein et al.’s levels of cognitive demand in supporting students’ mathematics heuristic problem-solving abilities

Wakhata, Robert; Mutarutinya, Védaste; Balimuttajjo, Sudi

doi:10.3389/feduc.2023.949988

ORIGINAL RESEARCH article

Front. Educ., 25 April 2023

Sec. STEM Education

Volume 8 - 2023 | https://doi.org/10.3389/feduc.2023.949988

Exploring the impact of Stein et al.’s levels of cognitive demand in supporting students’ mathematics heuristic problem-solving abilities

Robert Wakhata¹^*

Védaste Mutarutinya¹

Sudi Balimuttajjo²

¹African Centre of Excellence for Innovative Teaching and Learning Mathematics and Science, University of Rwanda, Kayonza, Rwanda
²College of Education, University of Rwanda, Rukara, Rwanda

The present study explored the impact of Stein et al.’s levels of cognitive demand (LCD) on evaluation and instructional methods in applying the knowledge of equations and inequalities to learn the topic of linear programming (LP). The framework provided by Stein et al. was used to map students’ LP cognitive demands. Students’ specific proficiency levels in solving LP tasks using Stein et al.’s LCD hierarchical framework were investigated. A mixed-method approach with a pre-test and post-test pre-intervention pilot study involving a non-equivalent control group design was applied. The participants were 175 grade 11 students from Mbale district, eastern Uganda. Two pre-interventional LP tests (pre-test and post-test) were administered to the students to examine their cognitive demands. This was followed by an intervention involving application of Stein et al.’s LCD in learning LP. The results showed that before pre-intervention, the performance of urban school’s average post-test scores was higher than that of the rural school. Students from the rural secondary school improved greatly relative to their peers from the urban school. Moreover, only 25.1% of students performed at the highest level of Stein et al.’s LCD (doing mathematics). The post-test scores were better relative to the pre-test (M = 56.51 ± 20.88 vs. 42.23 ± 22.49; p < 0.05). Overall, there was a statistically significant difference between students’ average grades in the pre-interventional pre-test and the post-test (Cohen’s d = 0.81 > 0.5), 95% CI [−18.00, −10.56]). Holding other factors constant, the significant differences in students’ scores were mainly due to the application of suitable tasks which were later mediated by the application of Stein et al.’s LCD instructional approach. This study, therefore, recommends that mathematics educators should effectively apply Stein et al.’s LCD to vary mathematics tasks given to students. This approach enhances students’ cognitive levels, supports students’ heuristic problem-solving abilities, critical thinking skills, and application of mathematics in real-life.

Introduction and the concept of cognitive demand

The objectives of learning mathematics and science are centered on students’ attainment of the three domains: cognitive, affective, and psychomotor (Lee et al., 2015; Sönmez, 2017). During the students’ holistic learning process, the three domains serve as primary educational goals. Moreover, the three learning domains complement one another in engaging, assessing, and evaluating student’s learning differences (Bloom et al., 1956; Krathwohl, 2002; Culliname, 2010). The challenge, however, is determining the content to be learned and how particular students’ learning outcomes are assessed. The teachers’ decision to choose a variety of tasks (to complement vast material from mathematics textbooks) based on student’s level of cognitive engagement and academic abilities, without sacrificing the consistency of learning outcomes, is critical. According to Stein et al. (2000), there is a connection between the tasks students are given and their relational conceptual understanding. Indeed, students are motivated to think innovatively and critically when they are given tasks with varying levels of cognitive demand.

Student’s levels of cognitive demand (SLCD) refer to their ability or inability to think and reason insightfully on various mathematical tasks during problem-solving (Stein and Lane, 1996; Stein et al., 1996, 2008; Henningsen and Stein, 1997; Smith and Stein, 1998). The levels of cognitive demand may vary according to students’ levels of cognitive development (Stein et al., 2000). Thus, appropriate mathematical tasks should be designed and implemented to reflect Stein et al.’s LCD (memorization, connection with procedures, connection without procedures, doing mathematics) (Smith and Stein, 1998). Furthermore, to adequately and appropriately address students learning differences and abilities, Smith and Stein (1998) proposed five integrated practices for enhancing students’ mathematical classroom discussions on challenging tasks. These include connecting, sequencing, selecting, monitoring, and anticipating. The use of these practices coupled with varied mathematical tasks, and characterized with suitable examples, may promote and improve student-centred heuristic problem-solving strategies and representations. Mathematics teachers may actively engage students to construct meaning from challenging problems and apply it in solving societal problems.

During lesson planning, lesson delivery, and lesson assessment, teachers play an important role by applying problem-solving heuristic approaches (see Polya, 2004; Suh, 2007), and introducing various tasks (as drills, assignments, assessment problems for group discussions, and so on) to the students to achieve particular learning objectives. In this respect, the teacher’s role is to shape the learning environment and students’ experiences based on the subject curriculum’s needs and expectations within the student’s zone of proximal development (Vygotsky, 1986). This may be accomplished by ensuring that students’ knowledge and skills are acquired and retained by the learning objectives. Consequently, students’ logical thinking and problem-solving abilities are enhanced. According to Suh (2007), mathematical proficiency is described as the impact on students’ social, economic, and national development in terms of conceptual understanding, procedural fluency, strategic competence, adaptive thinking, and productive disposition, rather than simply passing mathematics at the school level. Proficiency is viewed as students’ ability or their inability in applying what they have learned (critical thinking and problem-solving) in real-life scenarios.

The concept of active learning and the heuristic problem-solving

Active learning approaches are those instructional strategies that promote students’ engagement and active participation in constructing knowledge and understanding. These strategies include hands-on activities, problem-solving tasks, etc. This approach involves learners’ critical thinking and/or collaborative performance on routine and non-routine tasks. Active learning tasks require learners to make their thinking explicit, allowing educators to gauge and understand students’ learning. The approach engages students in deep and broader learning. It also builds the learners’ higher-order thinking skills. Learners develop a positive attitude toward learning as opposed to learners being passive listeners. The rationale is to develop students’ higher-order thinking skills (analysis, synthesis, and evaluation) and prepare them to apply mathematics in real-life scenarios. Studies (e.g., Prince, 2004; Freeman et al., 2014) have shown that students in typical active learning classrooms perform better than those taught conventionally. This is because learners have an opportunity to reflect, conjecture, or predict outcomes, and to share and discuss their concepts with teachers and their peers to activate and re-activate their cognitive processes. Active learning helps students to reflect on their understanding by encouraging them to make connections between prior mathematical knowledge and linking it to the learning of new concepts.

Unlike routine mathematics tasks that are usually conceptual and are answered by applying clearly defined and generally acceptable mathematical rules and principles, non-routine mathematical tasks involve the use of cognitive and meta-cognitive strategies (Mogari and Chirove, 2017). These strategies do not necessarily guarantee the solution to the problems but may help to establish effective procedures for finding approximate solutions (Abel, 2003). During problem-solving, one may choose the path that seems to result in some progress toward the goal. Such a rule is an example of a heuristic. A heuristic refers to a rule of thumb that serves as a guide to problem-solving processes (Abel, 2003). Research (e.g., Lester, 2013; NCTM, 2014; Mogari and Chirove, 2017; Chong et al., 2018) shows that non-routine mathematical problems are cognitively more challenging and demanding than routine tasks.

However, an extensive body of empirical studies (e.g., Tsamir and Almog, 2001; Bicer, 2007; Hj, 2010; Almog and Ilany, 2012; Jupri and Drijvers, 2016; Makonye, 2016; Makonye and Fakude, 2016; Mukuka et al., 2020) conducted in different contexts and settings have demonstrated that students’ difficulties in learning mathematics are caused by multiple factors. The factors are categorized as those stemming from students’ cognitive, affective, and psychomotor domains. Specifically, some study findings revealed that the learning environment, students’ misconceptions and errors, and teachers’ instructional and assessment practices account for most of the students’ difficulties in learning specific mathematics content. In Uganda, the situation is not different as reported by Uganda National Examinations Board (UNEB) (UNEB, 2016, 2018, 2019, 2020). Uganda National Examinations Board regularly reports poor performance in mathematics at the Uganda Certificate of Examinations (UCE) in comparison to other subjects. Specifically, LP mathematics word problems are one of the “hardest” and amongst the least-performed mathematics topics at UCE (UNEB, 2016, 2018, 2019, 2020). From the aforementioned research studies and reports, LP is cognitively demanding to students, and challenging for teachers during instruction due to limited learning materials to aid students’ understanding.

Indeed, students have always struggled to grasp the basic concepts of equations and inequalities which inherently limit their conceptualization of LP word problems. Different studies (e.g., Tsamir and Almog, 2001; Almog and Ilany, 2012; Jupri and Drijvers, 2016; Kenney et al., 2020) have been conducted in different contexts and settings to understand the causes and challenges of learning equations, inequalities, and LP. In the Ugandan context, the situation is not different as indicated in UNEB reports that the topic is hurriedly taught toward national examinations (for the sake of syllabus completion). Consequently, students do not adequately conceptualize and appropriately link the basic concepts of equations and inequalities to fully understand LP. It appears that some teachers have not fully addressed students’ LP perennial concerns and challenges by identifying sources of their difficulties in LP to provide suitable remedies. Although the above studies and other related findings (e.g., Opolot-okurut, 2010) show that students’ mathematical difficulties are a consequence of the affective domain, this paper explored students’ learning difficulties in LP through the lens of students’ LCD.

In human decision-making, heuristics are integrated into problem-solving (PS) and are referred to as exploratory PS techniques (Gurat, 2018). Heuristics are approaches to PS and encompasses students’ active involvement and self-discovery through experience as guided by educators. For instance, finding an optimal solution (e.g., to LP tasks) may be hard or impractical. The active learning heuristic PS strategies may be handy in finding satisfactory optimal solutions. According to Polya (2004), PS heuristics involve understanding the problem, devising the plan, carrying out the plan, and looking back. The implication is that to adequately solve any task or problem, learners should first understand the verbal statement of the problem (what is known? unknown?) and develop the desire to find the solution as a prerequisite step and a foundation for PS. This step is inevitable in the sense that failure to understand the problem yields wrong optimal solutions. However, some students may be faced with cognitive challenges of converting contextual information into conceptual understanding. Teachers should adequately address this challenge before solving non-routine LP word tasks. This may minimize misconceptions and errors (King, 1991). Heuristics are, therefore, basic decision-making processes for PS. Once the problem has been understood, it is transformed into schemas by relating to other auxiliary problems solved previously. This may be followed by writing correct procedures, and relating the known to other unfamiliar tasks. Then, check, verify, and prove that the steps are correct and finally examine the solution by checking the arguments, and investigating whether or not the method can be applied to solve related tasks.

Educators who apply the heuristic PS strategies are likely to develop students’ critical thinking and problem-solving skills (Hoon et al., 2013; Albay, 2019). Research has shown that students who are actively involved in PS achieve more, work collaboratively, and retain more (Klang et al., 2021). The student’s active learning process integrated with group work increases their conceptual learning, interest, and positive attitude toward what they learn. Other benefits of the heuristic PS approach include: students’ active participation in form of exploratory learning, arousing positive attitude toward specific mathematics content, development of student’s communication and social skills, fostering teamwork, and consequently addressing individual students’ learning differences (Gurat, 2018; Kigamba et al., 2021). In the context of learning LP, students’ application of effective PS strategies may yield optimal solutions. In this research, active learning heuristic problem-solving is contextualized to refer to the application of graphs, problem-solving, and Newman Error Analysis prompts for effective learning of LP.

While different strategies (e.g., curriculum change, continuous professional development, etc.) have been, and others are being undertaken by Ugandan educational stakeholders to mitigate students’ difficulties in learning mathematics (equations and inequalities), the students’ cognitive levels must be taken into consideration. Thus, there is a need to evaluate mathematical learning activities to improve, promote and foster students’ problem-solving skills, conceptual understanding, skills retention, and application of requisite knowledge in new situations. The students’ conceptual, procedural, and logical understanding of LP can be demonstrated by solving non-routine tasks of high-order thinking (doing mathematics). The justification is that the objective of studying mathematics in the Ugandan lower secondary school curriculum is to help students acquire and develop critical thinking skills, innovativeness, and problem-solving skills (NCDC, 2018). The main goal is to foster and support students’ mathematical cognitive levels. With the above context in mind, this study explored the impact of Stein et al.’s levels of cognitive demand in supporting students’ problem-solving heuristic approach. Effective application of Stein et al. LCD hierarchical framework may improve and retain students’ LP problem-solving. Specifically, this study mainly focused on answering the following research question: How does Stein et al.’s LCD support the heuristic problem-solving instructional approach when learning LP? It was hypothesized that teachers can gain insight into the application of the heuristic problem-solving strategies to enhance and support students’ understanding of LP concepts and hence boost their mathematical proficiency.

The theoretical framework

The framework provided by Stein et al. (2000) was used to map students’ LP cognitive demands. Cognitive demand was described by Stein et al. as “the kind and level of thinking required of students to successfully engage with and solve the task” (Stein et al., 2000, p. 11). Stein et al. interpreted these levels as problem-solving strategies. According to Stein et al., teachers should take into account different levels of cognitive demand with varying mathematics tasks given to students (Henningsen and Stein, 1997). The authors reasoned that students’ mathematical proficiency and competency are determined by the tasks they are given during instruction. Mathematics tasks at the lower cognitive stage (memorization level), for example, must be different from those at the highest cognitive level (doing mathematics). During task review, this approach supports teachers’ instructional activities and approaches (what is learned, how it is learned, and when it should be learned) following Stein et al.’s LCD. In the context of learning LP, students should first understand and appropriately apply the knowledge of equations and inequalities in solving LP non-routine mathematics word problems.

The practical aspect of Stein et al.’s LCD lies in its implementation (Henningsen and Stein, 1997). Indeed, if students are only exposed to memorization tasks, they may not be able to adequately master non-routine high-level tasks that require critical thinking skills (doing mathematics). Thus, as students advance through their academic stages, teachers need to adjust and involve them in answering high-level tasks from the beginning to improve their problem-solving abilities and skills. Stein et al. (2000) suggested four levels of cognitive demand: two lower-level demand tasks (memorization and procedures without connection to concepts) and two higher-level demand tasks (procedures with connections and doing mathematics). According to Stein et al., students’ proficiency in “doing mathematics” tasks may improve their problem-solving abilities especially in solving non-routine tasks. In this case, cognitive level characteristics (see Appendix 1) formed a framework for evaluating individual students’ levels of cognitive demand in learning LP tasks. These characteristics are important in the sense that they highlight specific cognitive levels needed for students to correctly perform LP mathematics tasks as well as specifying the students’ cognitive level(s) at the time the LP-test tasks (see Appendix 2) were administered.

Several empirical findings (e.g., Henningsen and Stein, 1997; Allsopp et al., 2007; Suh, 2007; Samuelsson, 2010; NCTM, 2014; De Jesus et al., 2015; Quintero and Rosario, 2016; Alex and Mammen, 2018) have emphasized the importance of instructional practices that cultivate and support students’ mathematical proficiency about Stein et al. LCD. Research has demonstrated that using multiple visual and symbolic representations in writing mathematical models from word problems is one of the appropriate and effective instructional practices. The above empirical findings are in agreement with Smith and Stein (1998, 1996) findings. Yet, students’ relational understanding of mathematics word problems necessitates the use of proper procedures. This may demonstrate a strong grasp of students’ mathematical problem-solving skills, abilities, and procedures. In solving and applying mathematics word problems to real-world scenarios, these procedures necessitate students’ mastery and ability to apply basic mathematical principles to explore and find solutions to non-routine mathematical tasks. This is due to the complexity of some mathematics tasks which appear challenging to some students. Therefore, using varied tasks and effective procedures coupled with students’ consistent practice may help learners to overcome misconceptions and errors in learning LP mathematics word problems.

Thus, educators who assign memorization tasks only to their students may only see mathematical tasks as cramming formulae and memorizing procedures (lowest cognitive level) compared to challenging tasks with conceptual and procedural connections that require high-order thinking (high-level tasks). Students, on the other hand, may cram and replicate concepts without using proper mathematical operations, conjectures, procedures, principles, and computations. In this context, students’ ability to comprehend and correctly apply the concepts of equations and inequalities in solving non-routine LP everyday world problems may minimize misconceptions and errors. Thus, students should be helped to understand and summarize information from mathematics word problems, write correct models (inequalities), correctly represent equations and inequalities graphically (on the same coordinate axes), identify the feasible region, and thus select suitable but multiple procedures for obtaining optimal solutions. In this research, Stein et al. LCD framework was applied to complement the heuristic problem-solving approach in analyzing students’ solution sketches to varying LP tasks. It was applied to identify students’ cognitive levels when solving LP tasks, as well as defining the specific cognitive levels they were working at.

Several studies (e.g., Henningsen and Stein, 1997; Allsopp et al., 2007; Cavey et al., 2007; Jones and Tarr, 2007; Stein et al., 2008; Samuelsson, 2010; Makonye, 2014; Budak, 2015; De Jesus et al., 2015; Pǎun, 2015; Parrish and Martin, 2020) have examined the importance of the Stein’s LCD framework. The findings show that each cognitive level predicts students’ proficiency in solving non-routine tasks. The above authors, Stein e ta l’s. LCD significantly contributes to teachers’ instructional practices by enhancing and fostering students’ logical, relational, conceptual, and procedural understanding of challenging mathematical concepts and tasks. Thus, the student’s cognitive levels were enacted during classroom instruction. While there are other educational curriculum taxonomies and scientific frameworks for assessing and promoting students’ cognitive levels (see Lee et al., 2015), they have been applied in different educational disciplines. As a result, these taxonomies may not satisfactorily measure specific levels of cognitive demand with students’ cognition in specific units or subject matter content. Therefore, the Stein et al. framework was appropriate for examining different levels of cognitive demand in learning and solving LP tasks. By effectively using this framework, different background factors (e.g., students’ age, the learning environment, students’ academic differences, previous intellectual awareness, etc.) that may account for students’ conceptual challenges in learning mathematics are mitigated.

The learning of linear programming mathematics word problems

Linear programming is one of the algebraic topics that require students’ understanding of basic mathematical principles and rules of equations and inequalities before the application of computer software for solving and optimizing more complex LP problems. Linear programming is a classical unit, “the cousin” of mathematics word problems which has gained significant applications in the last decades in mathematics, science, and technology (Romeijn et al., 2006; Colussi et al., 2013; Parlesak et al., 2016; Aboelmagd, 2018) because it links theoretical to practical mathematical applications. The topic provides elementary modeling skills (Vanderbei, 2014).

Previous empirical studies have revealed that LP and/or related concepts are not only difficult for learners but also challenging to teach (Awofala, 2014; Goulet-Lyle et al., 2020; Kenney et al., 2020; Verschaffel et al., 2020). Research shows that different factors account for learners’ challenges in mathematics word problems (Ahmad et al., 2010; Haghverdi et al., 2012; Heydari et al., 2015). The challenges range from students’ comprehension of word problem statements, and their attitude toward the topic, to their transformation from conceptual to procedural knowledge and understanding. Learners’ attitudes toward solving algebraic word problems should, therefore, be investigated and integrated during classroom instruction to help educational stakeholders provide appropriate and/or specific instructional strategies.

Solving LP tasks (by graphical method) is one of the topics taught to 11th-grade Ugandan lower secondary school students (NCDC, 2008, 2018). Despite students’ general and specific learning challenges in mathematics, the objectives of learning LP are embedded within the aims of the Ugandan lower secondary school mathematics curriculum. Some of the specific aims of learning mathematics in Ugandan secondary schools include enabling individuals to apply acquired skills in solving problems of the community, instilling a positive attitude toward productive work, and a strong respect for the dignity of labor and those who engage in productive labor activities; develop a positive attitude toward learning as a lifelong process…” (NCDC, 2018). Generally, the learning of LP aims to emphasize students’ problem-solving abilities, application of prior algebraic conceptual knowledge, and understanding of linear equations and inequalities in writing models from word problems, and in real-life-world problems. The topic of LP is also aimed at equipping learners with adequate knowledge and skills for doing advanced mathematics courses beyond the minimum mathematical proficiency (at the lower secondary school level).

However, for the last three decades, and every academic year, the Uganda National Examinations Board (UNEB) highlights students’ strengths and weaknesses in previous examinations at the Uganda Certificate of Education (UCE). The consistent reports (UNEB, 2016, 2018, 2019, 2020) on previous examinations on the work of candidates show that students’ performance in mathematics is not satisfactory, especially at the distinction level. In particular, previous examiners’ reports show students’ poor performance in mathematics word problems. The examination reports further revealed numerous students’ specific deficiencies in LP. Students’ challenges in LP stemmed from question comprehension and the formation of wrong linear equations and inequalities from the given word problem in real-life situations. Thus, wrong models derived from questions may result in incorrect graphical representations, and consequently wrong solutions and interpretations of the optimal solutions. These challenges (and others) may hinder or interfere with students’ construction of relevant models in science, mathematics, and technology. Moreover, learners have consistently demonstrated cognitive obstacles in answering questions on LP, while others elude questions on this topic during national examinations. Noticeably absent in all the UNEB reports are specific factors that influence students’ weaknesses and challenges in learning and solving LP tasks. To achieve the purpose of this research, the study aims to:

i. Compare and contrast students’ heuristic problem-solving abilities based on Stein et al.’s LCD.

ii. Describe and define specific cognitive levels students performed different LP tasks.

iii. Examine the relationship between the application of Stein et al.’s LCD and students’ heuristic problem-solving abilities in learning LP.

Methodology

Research design

This was part of a large study that investigated the effect of the heuristic problem-solving approach on students’ achievement and attitude toward learning LP word problems. The study adopted a mixed-method approach to gain a deeper and broader understanding. An explanatory pre-intervention pre-test, post-test, non-equivalent control group research design was adopted. This design enabled researchers to benefit from the advantages of triangulation by collecting both quantitative and qualitative data. The findings reported in this study were purely the comparison of performance for the rural school and urban school before intervention, and thereafter, this was later followed by the application of Stein et al.’s LCD as a problem-solving heuristic approach.

Participants

The research participants were 175 11th-grade students (106 females and 69 males) whose ages ranged between 16 and 20 years (M = 17.26, SD = 0.80). The age difference in the sample size was attributed to students repeating previous classes (grade 7 – grade 10) or were repeating the current class (grade 11). The 11th-grade class (locally called senior four) is the fourth and terminal class in the lower secondary school Ugandan education system. In this particular class, students are assessed by the Uganda National Examinations Board (UNEB), the national examining body leading to the award of the Uganda Certificate of Education (UCE). Subsequently, UCE certificates are a requirement for admission to the Uganda Advanced Certificate of Education (UACE) and admission to many other certificate courses. The students came from two mixed public rural (61 females and 37 males) and urban (45 females and 32 males) secondary schools in Mbale district, eastern Uganda. The schools were chosen purposively with similar characteristics, and students’ intact classes were chosen at random by using a flip of a coin. The heuristic problem-solving approach was applied to the rural secondary school to examine their levels of cognitive demand based on Stein et al. while students from the urban secondary school learned conventionally. At the time of data collection, the participants were completing the mathematics syllabus ahead of the national examinations for the academic year 2020/2021. Before research participants were requested to assent and consent, permission was granted by relevant authorities and educational stakeholders. They voluntarily agreed to participate in this study.

Materials

The achievement tests for the pre-interventional pre-test and post-test included LP-related tasks taken from mathematics textbooks. Specifically, questions on solving equations, inequalities, and formulation of equations and inequalities from word problems (see Appendices 1, 2). The standardized LP pre-interventional pre-test items were integrated with prerequisite concepts for learning LP as well as routine and non-routine LP mathematics tasks. The reason for administering the two tests was since the questions require different metacognitive abilities and were based on different levels of Stein et al. cognitive demand.

To ensure the suitability of LP tests in terms of face and content validity, this research followed Gay et al. (2016) recommendations. Six senior mathematics educators validated LP-related tests before test administration (2 expert secondary school teachers, 2 lecturers at teacher training colleges, and 2 university lecturers in mathematics education). All six experts had extensive mathematics pedagogical knowledge and experience in teaching mathematics of at least 15 years. The expert ratings helped to align LP tasks to reflect the curriculum content and per Stein et al. LCD. The LP tasks were pilot tested with 25 students outside the study sample (r = 0.84) to examine their clarity and suitability following SLCD (Lester and Bishop, 2014). The responses and feedback from the pilot study and the expert recommendations were used to modify LP-related tasks after a detailed review (see Appendices 1, 2). For instance, questions 1–3 on the lowest levels of cognitive demand (see Appendix 2) provided the necessary knowledge for answering LP tasks, whereas questions 4 and 5 on the highest levels of cognitive demand are typical non-routine LP tasks. In addition, to assess the quality of learning in the two schools, an observation scale adopted from Shafer et al. (1997) was used to consistently observe and record classroom activities, and students’ learning outcomes which were also audio and video recorded.

Procedure

Stein et al. (2000) provided a Task Analysis Guide (TAG) with basic characteristics at each stage that teachers can use to distinguish mathematical tasks. If a TAG is used effectively, the teachers’ classroom instructional practices can be improved in relation to the mathematics curriculum, teachers’ responsibilities, and students’ proficiency. Using a TAG meaningfully guaranteed the connection between the teachers’ activities and the learning outcomes. “After teachers learned about the framework, they began to use it as a lens for reflecting on their instruction and as a shared language for discussing instruction with their colleagues” (Stein et al., 2000, p. 4). The teachers’ difficulties, on the other hand, are in implementing tasks with a higher degree of cognitive demand. The authors further noted that even after retaining the tasks, there was a decrease in the degree of cognitive demand due to teachers’ ineffective pedagogical approaches. “The absence of this type of support contributed to the decline in the task from setup to implementation” (pp. 119–120).

The application of Stein et al. LCD in learning LP tasks was assessed using a pre-interventional pre-test, post-test, non-equivalent control group study design. A multistage cluster sampling method was used to choose secondary schools. All grade 11 students from secondary schools for the school academic year 2020/2021 constituted the sampling frame. The sample consisted of intact classes from rural and urban secondary schools. To minimize internal and external validity threats, the intact classes were randomly selected to investigate and/or explore Stein et al.’s LCD in learning LP. The results collected using this design may correlate with those from an experimental design (Fraenkel et al., 2011).

Later and by a toss of a coin, the rural secondary school was assigned to the treatment group while the urban secondary school was assigned to the comparison group. For the treatment group, the heuristic problem-solving instruction was applied during the learning process while students in the comparison group were taught conventionally. All students from both groups were subjected to a pre-test to assess their previous cognitive levels in LP. This was followed by an intervention in a rural secondary school where teachers had been trained to strictly adhere to the problem-solving heuristic approach of learning LP using Stein et al.’s LCD approach. The classroom instruction and observation of the learning outcomes were conducted during normal school working hours without altering the school timetable. Specific guidelines were given to the respective teachers and research assistants to strictly follow and ensure uniformity in implementing the heuristic approach. The entire learning process and supervision were monitored by the principal researcher assisted by four research assistants. Finally, all students sat for the post-test after the pre-test as a pre-intervention learning strategy that lasted for approximately 4 weeks. The students’ paper and pen LP solution sketches arising from the pre-intervention achievement tests were collected, scored, and analyzed based on the levels of cognitive demand.

Two open-ended tests, the pre-test, and the post-test, were developed by the researcher using questions from curriculum materials (textbooks), validated by mathematics experts. The pre-test (Appendix 1) was administered to students for 80 min as a pre-interventional evaluation and assessment examination. The purpose of the pre-interventional pre-test examination was to assess previous students’ Stein et al. LCD in solving LP tasks by graphical method. A marking guide was developed and designed with specific rubrics against solution sketches for scoring students’ written responses. Finally, all students sat a pre-interventional post-test (Appendix 2) for approximately 40 min (that lasted for approximately 4 weeks) to assess their cognitive demands mediated by students’ heuristic problem-solving strategies. The students’ paper and pen solution sketches arising from the two tests were collected, scored, and analyzed based on Stein’s levels of cognitive demand.

Expert teachers were gathered in one school to mark students’ handwritten answer scripts using a harmonized marking guide. We used a “conveyor belt scheme,” in which one expert teacher marked one question only before passing the answer script to the other expert. After that, 10% of the scripts were randomly sampled (using systematic random sampling) and moderated by one expert teacher (UNEB examiner) to assess the quality of marking. All errors and misconceptions omitted and/or committed by students were noted on their answer scripts. For consistency and comparability, all scores were translated to 100%. Any inconsistencies that arose during the marking and grading process were resolved. The main goal was to eliminate bias and deviation in marking students’ scripts if all of them were marked and graded by the same expert. Finally, the results of the pre-test were recorded for subsequent analyses.

The remedial lesson followed and involved the application of the heuristic problem-solving approach, a modified instructional and assessment approach to examine specific cognitive levels in learning LP. According to Polya, there are four steps to mastering problem-solving (Anderson et al., 2001). “Understanding the problem, Devising a plan, Carrying out a plan, and Looking back” (Polya, 2004, pp. 6–14). The heuristic problem-solving remedial classes lasted about a month (from mid-January to mid-February 2021). All students from the experimental group received 16 h of remedial instruction (4 h a week). The principal researcher oversaw and arranged the remedial activities with the respective teachers in the two schools assisted by the four research assistants. Specific activities included administering, assessment, and evaluation of LP pre-interventional pre-test and post-test. The observation of general students’ learning (introduction of LP concepts, lesson development, and evaluation), and identification of learning gaps for teachers by critiquing the lesson, not individual teachers was implemented to align the learning activities according to Stein et al.’s LCD TAG.

The students were taught solving LP tasks using Polya’s problem-solving heuristic approach. To create a post-test (see Appendix 3), the pre-interventional pre-test LP test items were made isomorphic equivalent. Finally, a pre-interventional post-test was administered to the same students after remedial lessons to assess and evaluate their specific cognitive levels. The same marking and grading techniques as for the pre-interventional pre-test were used for the pre-interventional post-test. The main aim was to establish if there existed a statistically significant effect on students’ cognitive levels between urban and rural areas. By using x to denote students’ percentage scores in LP the pre-test and post-test, the cognitive levels were categorized numerically as memorization- $(0 \leq x \leq 25)$ , connection without procedures ( $26 \leq x \leq 50),$ connection with procedures $(51 \leq x \leq 75),$ and doing mathematics $(76 \leq x \leq 100) .$

Data analysis and findings

SPSS software (version 26) was used for data analysis. Both descriptive and inferential statistics were used to analyze data. Specifically, the mean and standard deviation were used to describe the data. For inferential statistics, data were analyzed using paired sample t-test at the default significance level of 0.05. The groups’ pre-interventional pre-test and post-test average scores were compared to examine if there were any improvements in post-test scores as a result of the remedial lessons and if the observed average differences were statistically significant. Before parametric tests, data for the pre-interventional pre-test and the pre-interventional post-test was graphically inspected to check the respective assumptions. The graphical visualization revealed that data were fairly normally distributed. This was a confirmation to perform parametric tests for instance paired samples t-test and other related analyses.

Students’ abilities based on Stein et al.’s LCD for rural and urban secondary schools

Table 1 below indicates the average percentage scores in the pre-test and post-test for the students from the two schools. The aim of comparing the mean scores was to see how similar the students in rural schools and their counterparts from the urban setting performed before the remedial instruction. If all other variables (e.g., students’ age, prior knowledge, the learning environment, etc.) were held constant, the treatment effect (difference in mean scores) between the two groups would be due to the remedial lessons conducted.

TABLE 1

Table 1. Pre-test and post-test scores for rural and urban groups.

The average percentage pre-test and post-test score gap between rural and urban secondary schools are 28.02 and 10.88, respectively. This was largely attributed to a variety of factors (e.g., the school characteristics, students’ previous academic knowledge, experience, etc.) that might have contributed to the above-average score differences. As a result, no conclusions could be drawn from the above pre-test equivalence analysis. The mere fact that students in a rural secondary school performed poorly on the pre-test (29.90%) was not sufficient to infer that they were operating at Stein et al.’s lowest level of cognitive demand. Although there was a substantial average difference in students’ grades from the urban secondary school on the pre-test (by 28.02%), the post-test average scores improved greatly. This was suggestive that the problem-solving heuristic remedial lessons contributed to a differential effect on students’ post-test average scores for the rural secondary school (21.82%) compared to the average score difference of students in an urban secondary school (4.68%). The questions were standardized and set based on Stein et al. levels of cognitive demand, integrated into the problem-solving heuristic approach.

Specific cognitive levels at which students performed different LP tasks

Students’ cognitive levels from both groups improved to the highest standard (procedure with connection tasks and some of them to doing mathematics), it was observed that students’ LCD in a rural secondary school changed greatly as a result of the heuristic remedial lessons relative to their counterparts in an urban secondary school. Indeed, most students from a rural secondary school exhibited proficiency and were proficient in solving LP tasks by connecting procedures learned previously to solve more complex non-routine LP mathematical tasks (see Appendix 3) compared to their peers from the urban school. It is possible that students who performed well in the pre-interventional post-test greatly improved in their cognitive levels.

From the results in Table 2, the students’ percentage cumulative proficiency in LP tasks for the pre-interventional pre-test was scored within the lowest level of cognitive demand (64.6%). The observed students’ responses showed that the lowest scores were attributed to several misconceptions and errors in students’ LP solution sketches (e.g., wrong inequalities, incorrect feasible region, wrong numerical optimized solutions, wrong graphical representations with wrong coordinates). This revealed that this category of students was unable to fully comprehend the steps involved in solving typical LP non-routine tasks. Some students showed inadequate conceptual understanding of LP-related tasks and were unable to apply the basic algebraic principles in finding solutions to the LP pre-interventional pre-test and post-test even after completing the topic and the syllabus in general. Just 35.4% of students’ solution sketches were within the upper categories of Stein et al.’s LCD (connection with procedures and doing mathematics). Furthermore, only 13.1% of students were able to work at the level of “doing mathematics,” which required them to explore and understand LP-related tasks from word problems, write inequalities, represent inequalities on coordinate axes, define the feasible region, and/or write optimal solutions. It was observed that the LP tasks were cognitively demanding to students due to their limited understanding of LP word problems. Students were unable to apply prior knowledge and understanding of equations and inequalities to solve LP and related tasks.

TABLE 2

Table 2. Academic evaluations of students based on Stein’s levels of cognitive demand.

However, after the application of the pre-interventional heuristic problem-solving remedial lessons, the student’s problem-solving abilities improved (from 35.4 to 65.7%). The results further showed that only 25.1% of students performed at the highest level of Stein et al.’s LCD of “doing mathematics” as opposed to their pre-test results (see also a bar graph in Figure 1). Another observation was that the pre-interventional post-test results showed that the percentage of students within the lowest level of cognitive demand in the pre-test reduced by approximately half just as those in the highest level of cognitive demand increased by approximately half. Generally, students from the rural secondary school showed proficiency in problem-solving by exhibiting their abilities when answering LP tasks during the post-test, with fewer errors and misconceptions relative to those they made in a pre-test.

FIGURE 1

Figure 1. Students’ responses to Stein’s levels of cognitive demand.

Relationship between the application of Stein et al.’s LCD and students’ heuristic problem-solving abilities in learning LP

From the results of the independent two sample t-test, students performed better in the post-test (𝑀 = 56.51, 𝑆𝐸 = 1.58, SD = 20.88) relative to pre-test (𝑀 = 42.23, 𝑆𝐸=1.70, SD = 22.49), (174) = −7.58, 𝑝 < 0.05, d = 0.81(>0.5) (Tables 3, 4). The student’s scores on LP tasks in the post-test were 0.81 standard deviations higher than in the pre-test. It is, therefore, evident that the effect of heuristic problem-solving remedial lessons accounted for 81% of the total variance. Further examination of the data revealed that, on average, the post-test scores were 14.28% higher than the pre-test scores (95% 𝐶𝐼 [−18.00, −10.56]). It is worth noting that there was a statistically significant difference between students’ average grades on pre-test and post-test. Moreover, by controlling for other factors, the observed effect is large enough (Cohen and Manion, 1994; Field, 2009) to conclude that the differences in students’ LP average scores were primarily due to the pre-interventional heuristic problem-solving remedial lessons administered.

TABLE 3

Table 3. Paired sample statistics.

TABLE 4

Table 4. Shows paired samples t-test.

Qualitative findings through classroom observation

The findings point to the best learning practices concerning the application of the heuristic problem-solving approaches in learning mathematics integrated with Stein et al. LCD. Although it was assumed and/or predicted that the learning in all the schools would be uniform, the heuristic problem-solving approach adopted by teachers from the experimental group yielded academic differences after its implementation. A uniform classroom observation scale was used to assess the learning across all the sampled schools. The research team conferenced with the subject teachers in the experimental group immediately the lessons ended to point out the strengths and weaknesses during the learning process and to suggest areas of improvement. Specifically, we based on lesson preparation and organization, lesson presentation, the interaction between teachers and students, teachers’ subject content knowledge and relevance, classroom management, lesson assessment and evaluation, to provide general comments. Generally, students in the treatment group exhibited active participation during problem-solving compared to those in the comparison group. It was observed that students from the experimental group were free to engage in active discussions between teachers and amongst themselves, and, were able to apply the problem-solving strategies in solving non-routine LP problems. Students’ academic individual differences were harmonized through group work. This consequently enhanced their academic and social relationships as they continued to perform non-routine LP mathematical and related tasks. One of the students noted that the heuristic problem-solving approach was more engaging. She echoed that sharing basic concepts amongst themselves demystified the myth that LP tasks are difficult to pass. This particular student’s post-test score had doubled (80%) relative to her pre-test score (41%).

Discussion

This research explored the impact of Stein et al.’s levels of cognitive demand as a problem-solving heuristic approach during evaluation and instructional practices while learning and solving LP tasks. Based on the results of this study, Stein et al. level of cognitive demand is an effective framework for teachers in improving their instructional practices and, at the same time develop and evaluate students’ problem-solving abilities. This study provides insight for the teachers and consequently recommends that mathematics teachers should apply this framework to develop students’ mathematical proficiency. The findings of this study are consistent with other empirical studies in providing opportunities for successful mathematics learning (Stein et al., 1996, 2008; Smith and Stein, 1998; Cavey et al., 2007; Jones and Tarr, 2007; Ottmar et al., 2015; Parrish and Martin, 2020). We now discuss the teachers’ experiences in evaluating students’ LP mathematics tasks concerning Stein et al.’s LCD, and the basic concepts that may ultimately provide opportunities for students to develop their LP conceptual and relational understanding. Indeed, when teachers implement and appropriately vary the cognitive demand of the respective LP tasks, they may promote students’ mathematical proficiency. This is because different tasks necessitate different approaches based on levels of cognitive demand (Stein et al., 1996).

The findings of this study showed that the problem-solving heuristic approach had a significant effect on students’ conceptual and procedural understanding of LP concepts. The students from the rural secondary school generally showed proficiency in solving LP tasks by relating the basic algebraic concepts of equations, inequalities, and LP. Generally, the problem-solving heuristic approach applied to the rural secondary school yielded significant results with regard to the students’ average scores relative to the conventional approach which was applied in the urban secondary school. The results further revealed that at least 80% of students were operating at the lowest level of cognitive demand (memorization and procedures without connections) before the remedial lessons. The qualitative investigations showed that majority of students lacked the fundamental algebraic principles needed to solve non-routine LP problems during the pre-test. Indeed, students must be proficient in basic arithmetic and algebraic operations in order to write inequalities from word problems, understand the difference between equations and inequalities, avoid errors in linear inequality rules, graphing rules, and writing the solution set. Students’ failure to comprehend LP mathematics word problems was observed as a serious weakness.

The observation scale adopted from Shafer et al. (1997) was used to highlight the students and the teachers’ learning gaps. It was observed that the majority of students misinterpreted the LP word problem (see question 3, Appendix 3). As a result, incorrect inequalities were written (see Figure 2). Yet, wrong inequalities led to incorrect graphs and optimal solutions. Some students wrote correct inequalities but when they were requested to explain their thinking and solution process, they could not tell the difference between dotted and solid lines used in plotting graphs of inequalities. Some students were unable to locate the feasible region in which the solutions could be optimized. The characteristics of Stein et al. levels of cognitive demand for high-level tasks are similar to the above LP features (procedures with connection tasks and doing mathematics tasks). These results are consistent with UNEB reports on UCE candidates’ performance in LP (UNEB, 2016, 2018, 2019, 2020) and, Almog and Ilany (2012) findings with regard to students’ errors and misconceptions in learning inequalities.

FIGURE 2

Figure 2. Showing students’ Vignettes in terms of misconceptions in solving LP problems.

The observed paper and pen solution sketches in pre-test and post-test showed that students’ LCD varied due to students’ academic background characteristics with limited or no significant differences with gender and school characteristics. When some students from the rural secondary school were interviewed, they testified that the application of the problem-solving heuristic approach helped them to relate prior concepts of equations and inequalities in learning LP. It appeared that these particular students needed to be reminded of the basic concepts prior to the learning of LP. Indeed, one student echoed that “the methods used by our teacher in learning linear programming have helped me grasp all the concepts in the topic.” Another student added that “I wish we were taught like this from the beginning when we enrolled in senior one.” The other student who happened to be a slow learner appreciated the teacher for taking into consideration students’ individual academic differences in learning this topic. He went further to explain that “the reason why many of us hate some topics in mathematics is due to methods used by some of our teachers who rush with few students during the lessons leaving many of us.”

The heuristic problem-solving approach which was applied in this study followed Polya’s problem solving heuristic framework for identifying students’ LCD and their relational abilities in solving LP tasks. Other researchers have used the Newman Error Hierarchical heuristic model to study and identify students’ errors and misconceptions in solving mathematics algebraic word problems (Clements, 1980; White, 2009; Singer and Voica, 2013). The authors used Newman literacy and numeracy prompts to teach students how to solve mathematics word problems in a systematic way. Reading, comprehension, transformation, process skill, and encoding are the stages of solving mathematics word problems. According to Singh et al. (2010), this model is sequentially accurate, with the first two steps determining whether or not the correct solution is obtained, and the last three indicating that students have grasped the problem-solving process. Failure to follow the above systematic model leads to unsatisfactory results. Newman literacy and numeracy prompts are outside the scope of this study. However, related procedures were applied in answering LP word problems including, identifying parameters, writing models, graphing the relationships on the coordinate plane, shading unwanted regions, and finally obtaining the optimal solution (Stein et al. highest level). These procedures are significant as students transition from secondary to university. This is perhaps why the topic of LP is taught in secondary school and university mathematics. Following Lyakhova and Neate (2021) recommendations, the above procedures may support teachers and students in learning LP and for obtaining further mathematical qualifications.

According to the constructivists, students construct their mathematical knowledge, understanding and meaning by natural thought through problem solving (e.g., Bostic et al., 2017) rather than memorization of external laws and concepts. Thus, the learning should be structured in such a way that it prepares students to employ critical thinking in the face of complex situations rather than memorization of solution sketches and procedures from previous mathematical problems and solutions. Indeed, and from results of independent two sample t-test, students performed slightly better in the post-test (𝑀 = 56.51, 𝑆𝐸 = 1.58, SD = 20.88) relative to pre-test (𝑀 = 42.23, 𝑆𝐸 = 1.70, SD = 22.49), (174) = −7.58, 𝑝 < 0.05, d = 0.81(>0.5). Thus, the teachers’ role is to help students who fail to comprehend mathematical concepts to think innovatively by avoiding cram work and/or rote learning.

Griffiths and Shionis (2021) have argued that students’ positive attitude toward particular mathematics content should be emphasized as this is witnessed practically when applying the concepts learnt. Attitude toward mathematics affect students’ comprehension of basic concepts and their relationship with the subsequent learning. Comprehension challenges may limit subsequent learning due to variation in students’ levels of cognitive engagement which is sometimes triggered by students’ stages of cognitive growth. Therefore, scaffolding may help students in learning new content. Empirical research (e.g., Cobb et al., 1992; Henningsen and Stein, 1997; Tsou, 2006) show that engaging students, devising effective procedures and applying prior conceptual knowledge in solving challenging tasks helps students to master mathematical concepts within their zone of proximal development.

Teachers should, therefore, choose and vary the learning methods, approaches and tasks to accommodate students’ individual academic differences as they understand student’s cognition patterns. Kempen (2021) and NCTM (2014) recommend peer instruction amongst students as this fosters active construction and retention of knowledge and skills. These findings are consistent with the Stein et al. levels of cognitive demand framework as outlined in the task analysis guide. Consequently, teachers gain insight in integrating students’ stages of cognitive growth and development during the learning process if they use effective instructional and evaluation methods. In addition, teachers should think outside the box and employ alternative techniques and/or practices (for example, group work, cooperative learning, etc.) to motivate, develop, enhance and encourage students’ problem-solving skills and their abilities. For students, the levels of cognitive development have a number of benefits. First, teachers create learning goals based on assignments, anticipating students’ shortcomings and devising appropriate strategies to solve them. Second, instead of focusing on memorization tasks only, if mathematics exercises are based on Stein et al. levels of cognitive demand, they help students to develop problem-solving skills generally, and, to demystify the abstract nature in solving LP mathematical tasks. This may propel students to justify, gain trust in, and be motivated to complete varieties of tasks with varying levels of cognitive demand.

Conclusion

The main goal of this research was to explore how teachers could apply Stein et al.’s LCD as a problem-solving heuristic approach to enhance students’ abilities in learning the topic of LP. Polya’s problem-solving heuristic framework was applied to achieve the purpose of this study. The teachers were trained on the best approaches of applying the heuristic problem-solving approach to foster and support Stein et al.’s LCD in learning LP. To do this, teachers identified varying students’ cognitive levels, for improving their mathematical skills and abilities, while at the same time correcting their misconceptions and errors. To enhance proficiency in problem-solving, this study found that the heuristic problem-solving approach involving Stein et al.’s LCD greatly improved students’ critical thinking and problem-solving abilities.

Generally, the heuristic problem-solving mediated by Stein et al.’s LCD had a significant effect on students’ conceptual understanding of LP mathematics tasks. The framework, in particular, serves as a guide for teachers to develop and support students’ specific tasks and solving them at each level of cognitive demand. The features of Stein et al. levels of cognitive demand at each stage of cognition are particularly important to both teachers and students in terms of optimizing classroom teaching during the learning process. This is done with the aim of finding clear learning gaps and correcting errors and misconceptions that may impair students’ problem-solving abilities. To gain more expertise, skills, and conceptual understanding in solving different mathematics tasks, students were encouraged to practice, master and implement Polya’s problem-solving heuristic approach on a regular basis. Generally, Polya’s heuristic problem-solving framework provided learning opportunities for students and teachers in cultivating and developing procedural and conceptual understanding and proficiency, as well as being successful in problem-solving and remaining centered when dealing with non-routine mathematics tasks.

The findings of this study also provide insight to educational stakeholders in applying various tasks to evaluate students’ LCD, as well as providing remediation and interventional studies aimed at changing students’ attitudes toward learning mathematics and LP in particular. This may serve as a lens for analyzing relationships between students’ mathematical achievement, and their attitude toward studying specific mathematics content, as a measure of students’ confidence, usefulness, and enjoyment in solving LP-related tasks. The findings showed that using Stein et al. LCD framework enhanced students’ high-order thinking and mathematical conceptualization in terms of their engagement and problem-solving abilities. This was achieved by making mathematical connections with students’ prior knowledge during the learning process relative to memorization tasks that required minimum cognitive analysis. In this research, Stein et al.’s LCD in learning LP tasks generally increased and improved. However, this study had some limitations due to the limited geographical scope (Mbale district and participants’ sample of two secondary schools). Thus, caution must, therefore, be taken when generalizing our findings. To compare, contrast, and/or confirm our analyses and conclusions, we recommend replication of studies on LP and in different settings and contexts. Thus, there is need for a holistic investigation into the problem-solving heuristic approach on students’ mathematical conceptualization in different geographical contexts (with a larger sample) and in specific topics (LP included). This is due to the fact that cognitive taxonomic analyses are never conclusive, but rather cyclic.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

This research was approved by the corresponding author’s University Research and Innovation Ethics Committee. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.

Author contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2023.949988/full#supplementary-material

References

Abel, C. F. (2003). Heuristics and problem solving. New Dir. Teach. Learn. 2003, 53–58. doi: 10.1002/tl.113