Skip to main content

ORIGINAL RESEARCH article

Front. Psychol., 05 October 2022
Sec. Educational Psychology
This article is part of the Research Topic Exploring the Psychology of Vocational Education: From the Perspective of Literacy Promotion View all 24 articles

Research on the cultivation of sustainable development ability of higher vocational students by creative thinking teaching method

\nXin-Zhu Li&#x;Xin-Zhu Li1Chun-Ching Chen
&#x;Chun-Ching Chen2*Xin Kang&#x;Xin Kang3
  • 1College of Design, National Taipei University of Technology, Taipei, Taiwan
  • 2Department of Interaction Design, National Taipei University of Technology, Taipei, Taiwan
  • 3School of Design, NingboTech University, Ningbo, China

Education for sustainable development (ESD) is an important guideline for United Nations Sustainable Development Goals. Students' creative thinking can be applied to various disciplines, promoting sustainable learning. Most of Taiwan's beauty and hairdressing technical education teachers mainly teach students to imitate, and students' works lack creativity and thinking. A total of 43 higher vocational college students participated, 23 of whom were in the experimental group using the creative thinking teaching method and 20 of whom were in the control group using the traditional teaching method. The results show that the creative thinking teaching method can effectively improve students' learning outcomes in the multimedia material creation course, including breaking through the limitation of thinking, putting forward different ideas and answers, and constantly innovating, to make the presented results more creative and meaningful. The creative thinking teaching methods solve students' trouble in creative problem solving, enhance students' problem solving and critical thinking skills, and improve students' involvement in the study.

Introduction

Sustainability has become an increasingly important topic. Education for sustainable development (ESD) is regarded as a key driving force for United Nations Sustainable Development Goals (SDG) (Rieckmann, 2017; Leicht et al., 2018; Zhou and Lee, 2022). Higher education has regarded sustainable development as an important education goal (Molderez and Ceulemans, 2018; Sidiropoulos, 2022). Especially, creative thinking is an essential and important part of modern sustainable education (Ocetkiewicz, 2021), but is underestimated in many formal educations, especially technical and vocational education.

Beauty and hairdressing is an important part of technical and vocational education. The training goal is to enable students to have professional knowledge and practical skills related to hairdressing and cultivate talents for the industry (Chen and Liu, 2019). Most of the Department of beauty and hairdressing courses are based on traditional teaching, which is combined with the content of the Vocational Training Bureau's skill test and lack of space to provide students with diversified thinking (Wu et al., 2014). In teaching, the teacher will first demonstrate and then let the students imitate, strengthen the students' professional ability in many exercises, and make works by observing, imitating, and then observing. When students imitate the works demonstrated by teachers and repeat practice during the course (Hsia et al., 2021), they are often limited by the standardized template framework, such as eye-brow angle, color, the position of blush, etc. The traditional teacher-centered teaching methods limit students' creative thinking ability and reduce their interest in learning. This is the current dilemma of beauty and hairdressing technical education, as shown in Figure 1. Therefore, beauty and hairdressing technical education in higher vocational colleges need to try to improve the current dilemma through different teaching methods.

FIGURE 1
www.frontiersin.org

Figure 1. Traditional teaching process of cosmetology teacher.

Many studies have pointed out that student-centered creative thinking teaching method can enhance students' creative thinking ability (Guilford, 1968; Torrance, 1972; Chou, 2004, 2013; Chen, 2006; Black et al., 2015). It is suggested that educators use non-traditional teaching strategies to interact with students to arouse students' interest in learning. Creative thinking skills can be integrated into various disciplines, and can also be used in daily life (Lai, 2006), such as music education courses (Hickey and Webster, 2001; Burnard, 2012; Kladder and Lee, 2019), fashion design courses (Black et al., 2015), English drama courses (Dora To et al., 2011), nursing education (Ku and Kuo, 2016), etc. It can be seen that the creative thinking teaching method can be applied in various disciplines, but there is a lack of application of creative thinking teaching in Taiwan's higher vocational education.

The implementation principle of creative thinking teaching is to continuously use various creative thinking strategies in the process of teaching, let students put forward multiple opinions, do not criticize students' ideas, and encourage autonomous learning and provide attempts (Morris, 2006; Lin, 2008; Huang, 2011). The positive interaction between teachers and students can effectively improve learning outcomes (Huang, 2011). Chen et al. (2022) results show that the students' imagination was significantly improved by being involved in the creative learning activities. Creative thinking skills in the process of teaching can effectively stimulate students' creativity, increase the interaction between teachers and students, and effectively enhance students' participation. There is reasonable evidence to suggest that providing diverse appropriate materials, tools, and other resources can stimulate students' creativity (Kennedy, 2010; Robson and Jaaniste, 2010; Davies et al., 2013). Through the above literature, we can find that the creative thinking teaching method is a student-centered teaching method that can effectively enhance students' creative thinking skills, learning motivation, and learning outcomes through different teaching strategies and materials.

Creativity and creative thinking courses have become the future development trend. Creative thinking is considered to be an important ability in the 21st century (Fadel and Trilling, 2010), creative thinking has become a necessary ability in the future and has been paid more and more attention. Creative thinking teaching changes the traditional teaching method, integrates diversified materials, enables students to put forward different questions through thinking, and then uses diversified materials to assist the practice of creation, which is more conducive to improving students' creative thinking ability.

Although there are lots of researches on creative thinking teaching, seldom are about beauty and hairdressing technical education. Therefore, this study will make up for this literature gap, provide empirical experience for the research of related disciplines, and also hope to arouse the reflection of vocational education teachers on the role and status of teachers in thinking teaching.

This study integrates the teaching method of creative thinking into the course of material creation, and explores the comparative study on the learning outcomes of the creative thinking teaching method and the traditional teaching method.

Literature review

Comparison between traditional teaching method and creative thinking teaching method

Traditional teaching methods and creative thinking teaching methods are different in idea, goal, mode, method, material resources, materials, aids, and evaluation strategies. The traditional teaching method is a teacher-centered narrative teaching method, in which teachers lead all teaching activities, instill knowledge into students, emphasize recitation of knowledge and facts, and treat students as passive receivers (Mitchell and Martin, 1997); Creative thinking teaching method is a student-centered teaching method, which emphasizes students as participants and masters, and uses creative thinking strategies to teach students to solve problems and creative thinking methods, to find inspiration and stimulate creativity from reasoning (Chen, 2006; Liu et al., 2007).

In the traditional teaching method, teachers control the teaching process, and the course limits students' creativity, thus students cannot bring their ideas into play through one-way teaching (Chen, 2006). Especially in the course of cosmetology, students copy the works demonstrated by teachers just like photocopiers. In contrast, the creative thinking teaching method aims to develop learner autonomy and independence (Jones, 2007). The teacher plays the role of facilitator rather than an instructor. The teacher's goal in the learning process is to guide students to new interpretations of the learning material, thereby 'experiencing' content, As Rogers' notion that “important learning is acquired through practice.” (Kraft, 1978). In the creative thinking teaching methods, students are the focus. Therefore, it can improve students' problem-solving skills and develop their creative thinking and critical ability (Su et al., 2017). In this way, students can generate ideas, put forward hypotheses, and apply their imagination to find alternative creative ideas (Tan et al., 2016). However, creative thinking teaching methods combine various teaching strategies and show that students play a major role in the study. The literature found that pedagogical practice is one of the key environmental features in shaping student creativity (Davies et al., 2013).

Creative thinking teaching method and learning outcomes

The teaching of creative thinking is that teachers use a variety of teaching contents, methods, and activities to cultivate students' creativity (Ku and Kuo, 2016). The creative thinking teaching method has four characteristics: (1) encourage the use of imagination, increase students' creative ability; (2) In learning activities, students are the main body, teachers give guidance in teaching, and do not monopolize the whole teaching time; (3) The learning environment should be free, safe and harmonious; (4) Teaching methods should focus on stimulating students' interest in learning, encouraging students to express their own ideas and accept different opinions and thinking, and not rush to make judgments (Lin, 1974; Cropley, 1997; Chen, 2006; Davies et al., 2013; Schweisfurth and Elliott, 2019).

In the implementation of creative thinking teaching, students often show adventurous, challenging, curiosity and imagination, and these characteristics are included in the emotional level, which are related to students' attitude, value, appreciation and motivation, and are called creative tendency (Chen, 2006). Thomas and Chan believed that creative tendency is one of the commonly used tools to measure teenagers' creativity, including risk-taking, curiosity, imagination and challenge (Thomas and Chan, 2013), which can reflect psychological state (Shi et al., 2013).

Cosmetology is a part of vocational education, and also belongs to the field of art and design. Besides training students' skills, the most important thing is to enhance their creativity. Some studies have shown that the use of multi-media materials and a wide range of resources and tools in the course can effectively enhance students' creativity (Beghetto and Kaufman, 2014; Al-Dababneh et al., 2019). Creative thinking teaching provides an environment for students to constantly think and create, and stimulates students' creativity. In the course of multi-media creation, students are guided to learn through concept establishment and technical guidance.

Learning outcomes assessments

Learning outcomes refers to the changes in learners' knowledge, skills, and attitudes after the end of the teaching activity. Wager believes that learning outcomes are the learning goal that teachers expect students to achieve (Wager, 2003). Learning outcomes can be used as the basis for evaluating students' performance, reflecting the merits and demerits of the teaching plan, enabling students to understand their own learning status, and can be used as the basis for teachers to improve teaching and students to improve learning. Brown believes that learning outcomes are the presentation of knowledge and skills learned after formal courses (Brown, 1981). Effective assessment for Learning can improve students' attention to the course and motivate students to participate in course activities (Wu et al., 2021a,b). Kirkpatrick proposed the Four-Level Training Evaluations Model to evaluate the learning outcomes of students, which are reaction level, learning level, behavior level, and result level (Kirkpatrick, 1975; Kirkpatrick and Kirkpatrick, 2006).

Kirkpatrick's four-level evaluation is the most widely used evaluation model for learning Outcomes (Lien et al., 2007; Mollahoseini and Farjad, 2012; Sidiropoulos, 2022). According to the theory of Kirkpatrick, we can know whether the teacher's goal is achieved through the teaching process and whether the students' expectations and needs for the course are met in the learning process. The level of learning satisfaction at the reaction level can affect the learning performance during training. It can be seen that it is very important to understand the students' learning satisfaction in measuring the learning outcomes. Through the evaluation, we can better understand the real benefits of students through the learning process, as well as the suggestions for teachers' curriculum and teaching methods.

Methods

In this study, learning outcomes are defined as the change of learners' cognition, affection, and skills through the learning and understanding of knowledge and the application of skills. According to Kirkpatrick's evaluation model after learning: reaction, learning, behavior, and result, students' learning outcomes are measured. Learning satisfaction is used as the “reaction level” evaluation, the creative tendency is used to evaluate the learning level of curriculum absorption, and the expert consensus is used to measure the behavior level of students' professional ability to apply what they have learned in the course. Since the “result level” refers to the changes in people's psychology, skills and abilities, which requires a long-term observation time process (Gagné, 1985). It is difficult to set an evaluation standard and measure time. Therefore, this study will not include the evaluation and verification of the “outcome level” and only conduct a cross-sectional study.

Research design

Based on the research objectives, this paper adopts the quasi-experimental design method of equal group post-test design, and carries out experimental workshops for the two groups, respectively using the creative thinking teaching method and the traditional teaching method. The teaching plans of both the experimental group and the control group were prepared by researchers. The subject of the course is to design a white mask with a length of 19.5 cm and a width of 10 cm. In terms of course materials, the control group used the same material package. Materials for the experimental group were selected by students in an open way according to their own creativity, as shown in Table 1.

TABLE 1
www.frontiersin.org

Table 1. Teaching design of experimental group and control group.

The subjects of this study were junior students of the department of applied cosmetology in southern Taiwan. The students were randomly divided into experimental groups and control groups. Group A was the experimental group. Group A total of 23 students (male 2, female 21) received the multimedia material creation course of creative thinking teaching method. The students were coded from SA1 to SA23. Group B was the control group and received the traditional teaching method of the multimedia material creation course. a total of 20 people (male 0, female 20). The students were coded from SB1 to SB20, as shown in Table 2. The two groups received eight 45-min courses.

TABLE 2
www.frontiersin.org

Table 2. Summary of participants.

Questionnaire design

The questionnaire is divided into two parts. The first part is to measure the response level (learning satisfaction) and learning level (creative tendency) of students' learning outcomes, with total 44 questions; In the second questionnaire, the teacher assessed the behavior level of students' learning outcomes (expert consensus assessment), a total of 12 questions. The Likert five-point scale was used to measure each question, (1) strongly agree; (2) agree; (3) neither agree nor disagree; (4) disagree; (5) strongly disagree.

Questionnaire 1 was distributed to the students after the course, which was used to investigate and evaluate the differences in learning outcomes between the two groups of students applying different teaching methods. Questionnaire 2 was distributed to the students after the material creation course, the students' work photographs were submitted to experts for consensus evaluation. The expert consensus scale adopts the creativity power scale in the field of art revised by Amabile in 1983 (Amabile, 1983). In the research of consensus scale, the raters are generally divided into three categories: college teachers, high school or primary school teachers, and professionals (Lai, 2006; Kousoulas, 2010; Shih, 2013; Amabile et al., 2018; Jeffries et al., 2018). In this study, experts are divided into the above three categories, including 2 teachers from colleges and universities, 2 teachers from high schools, and 2 experts from the industry. Each expert independently evaluates students' works by using the consensus rating scale.

Through the questionnaire to evaluate the impact of creative thinking teaching method on students' learning outcomes, the literature shows that the learning outcomes can be divided into reaction level (curriculum content satisfaction and teaching method satisfaction), learning level (adventure, curiosity, imagination, challenge), behavior level (creativity, aesthetic attraction, technical advantages), and result level, this is a cross-sectional study, and the resulting level is not in the scope of this study.

Questionnaire 1: Students' feedback

Questionnaire 1 is divided into the following two parts:

The first part is the reaction level of learning outcomes, including content satisfaction and teaching method satisfaction. The questionnaire was prepared concerning the Creativity Training Course Teaching Feedback Form (Chou, 2013), Learning Response and Learning Interest Questionnaire (Huang, 2011), Hairdressing Course Creativity Reflection Learning Response and Learning Interest Scale (Tseng, 2009), and Home Economics Course Learning Response Scale (Chan, 2004), Learning Response Questionnaires (Wu, 2002), Teaching Response Scale (Wei, 2001), and theories proposed by Kirkpatrick and Kirkpatrick (2006).

The second part is the learning level, including risk-taking, curiosity, imagination, and challenge. This part of the questionnaire was prepared by referring to the Williams Creative Tendency Scale revised by Lin and Wang (1994) and the Creative Tendency Scale revised by Hung (2011). Students' learning outcomes were assessed from four aspects: adventure, curiosity, imagination, and challenge. A total of 20 questions were asked.

Questionnaire 2: Teacher evaluation index

Questionnaire 2 is the behavior level, including creativity, aesthetic appeal, and technical advantages. The consensus assessment technique proposed by Amabile is developed based on appropriate social-psychological methods. It refers to that when experts in a certain field look at products or clearly observable reactions, able to evaluate creativity (Amabile, 1983; Chang, 2013; Amabile et al., 2018). According to the characteristics of this study, 12 questions of the expert consensus rating scale proposed by Amabile were selected, which were divided into four questions of creativity group (creativity, novel use of materials, novel idea, and significance of effort). Aesthetic attractiveness group four questions (aesthetic attraction, display value, preference, and pleasing shape configuration); The technical value group uses four questions (technical advantages, overall organization, expressiveness, and fineness) to evaluate student works (Amabile et al., 2018), as shown in Table 3.

TABLE 3
www.frontiersin.org

Table 3. Expert consensus assessment (behavioral level).

In the expert consensus assessment section, a validity analysis was carried out in order to determine the quality of the measuring instrument. The procedure used was expert judgment, by which an assessment of each item's quality and relevance was carried out by 4 expert judges. The expert validity of the questionnaire was evaluated by 2 higher vocational teachers and 2 college professors according to the research direction and the content of the questionnaire.

There are 12 questions in which 6 experts evaluate students' work, and take the average of 6 experts to assess the behavioral aspect of learning outcomes of the two groups. The experts were coded from E1 to E6.

43 questionnaires were collected from the two groups, of which 23 were collected from group A (experimental group), and 20 from group B (control group). The questionnaire recovery rate was 100%.

Research hypotheses

This study explores the differences in learning outcomes between creative thinking teaching and traditional teaching, based on the above relevant literature and theoretical research, we propose the following hypotheses according to the four-level evaluation model proposed by Kirkpatrick (Kirkpatrick, 1975; Kirkpatrick and Kirkpatrick, 2006), which are response level, learning level and behavior level, respectively, and the result level is presented in the research result part.

Reaction level

The integration of creative thinking teaching method into the curriculum of all disciplines will improve students' learning satisfaction. In terms of students' reaction level, students hold a favorable and positive attitude toward course activities, course content and teaching methods (Li et al., 2011; Hu et al., 2016; Huang et al., 2021). Therefore, the hypothesis at the reaction level of this study is as follows:

H1a: The creative thinking teaching method is positively related with the learning satisfaction in course content satisfaction.

H1b: The creative thinking teaching method is positively related with learning satisfaction in Teaching method satisfaction.

Learning level

Some studies have found that the creative thinking teaching method has a significant impact on all aspects of creative tendency (Chen, 2003; Hung, 2004), but there are also studies show that there is no significant impact (Liu et al., 2020). It can be seen that the creative thinking teaching method can affect students' creative thinking ability, but it is not clear whether the creative thinking teaching method can significantly improve students' creative tendency. Therefore, this study deduces the following hypotheses based on the above relevant literature:

H2a: The creative thinking teaching method is positively related with the adventure of the creative tendency.

H2b: The creative thinking teaching method is positively related with the creative tendency of curiosity.

H2c: The creative thinking teaching method is positively related with the creative tendency of imagination.

H2d: The challenge of creative thinking teaching method is positively related with creative tendency.

Behavior level

Expert consensus assessment is a common way to assess students' learning outcomes, which has good consistency in student assessment (Lai, 2006; Shih, 2013). Based on relevant studies, the expert consensus assessment is divided into three parts, namely creativity, technical advantage and aesthetic appeal, respectively (Shih, 2013), we propose research hypotheses from the above three aspects:

H3a: The creativity of expert consensus assessment is positively related with the creative thinking teaching method.

H3b: The aesthetic sensibility of expert consensus assessment is positively related with the creative thinking teaching method.

H3c: The technical advantage of expert consensus assessment is positively related with the creative thinking teaching method.

Figure 2 shows the research hypothesis framework.

FIGURE 2
www.frontiersin.org

Figure 2. Research hypotheses.

Results

Based on the collected questionnaire data, this part analyzes the research questions and hypotheses and discusses the correlation between the creative thinking teaching method and learning outcomes. To analyze the reliability, Cronbach's Alpha was utilized. The Cronbach's Alpha value of 0.94, coefficients found obtained values higher than 0.70, and this indicates an acceptable degree of reliability between the items that shape the latent variable, as shown in Table 4.

TABLE 4
www.frontiersin.org

Table 4. Assessment of convergent reliability.

Result 1: Students' feedback

According to Levene's homogeneity test of variation, whether there is a significant difference in the variation of the reaction level of learning satisfaction (satisfaction of course content and teaching method) of learning outcomes between control group B and experimental group A can be determined. Then, whether there is a significant difference in the mean value of t-test and significance test, and all data were analyzed using IBM SPSS 22.0.

An independent samples t-test was adopted to test for differences between the two groups of students about course content satisfaction and teaching method. The results are as follows: course content satisfaction (t = 0.403; p > 0.05), teaching method (t = 4.801; p = 0.000 < 0.05), indicating that there was significant difference between the creative thinking teaching method and course content satisfaction in the reaction level, and there was a significant difference between the learning satisfaction level of creative thinking teaching method and learning satisfaction, as shown in Table 5.

TABLE 5
www.frontiersin.org

Table 5. Expert consensus assessment (behavioral level).

A difference analysis was conducted for experimental group A vs. control group B creative tendency. An independent samples t-test was adopted to test for differences between the two groups of students about adventure, curiosity, creative thinking, and challenge. The results are as follows: adventure (t = 1.685; p > 0.05), curiosity (t = 0.643; p > 0.05), creative thinking (t = 0.624; p > 0.05), challenging (t = 0.658; p > 0.05), indicating that there was no significant difference between the creative thinking teaching method and the tradition-al teaching method in the level of creative tendency (adventure, curiosity, imagination, and challenging), as shown in Table 6.

TABLE 6
www.frontiersin.org

Table 6. Independent sample t-test for two groups of creativity tendency.

2 weeks after the end of this experiment, two groups of students experienced the teaching method of the opposite group. In this way, all the students experienced both the traditional teaching method and the creative thinking instruction method. Then, students' feedback and what they have gained were asked. Students SA5, AS11, SA13, SA22, SB4, and SB10 from the creative thinking instruction group said, they used different thinking methods to create more narrative and creative works. Besides, students SA2, AS18, SA20, SB1, SB15 said, both of these two teaching methods could inspire their learning motivation but courses in creative thinking instruction method were more interesting. Importantly, students SA1, AS15, SB17, and SB20 said, the traditional teaching method was boring, and they preferred the creative thinking instruction method. These feedbacks were the same as what data had been presented.

Result 2: Teacher evaluation index

A difference analysis was conducted for experimental group A vs. control group B expert consensus. An independent samples t-test was adopted to test for differences between the two groups of students in creativity, aesthetic sensibility, and technical advantage. The results are as follows: creativity (t = 2.887; p < 0.05), technical advantage (t = 2.640; p < 0.05), aesthetic sensibility (t = 2.484; p < 0.05), indicating that there was a significant difference between the creative thinking teaching method and the traditional teaching method in the level of behavior (creativity, aesthetic sensibility, and technical advantage), as shown in Table 7.

TABLE 7
www.frontiersin.org

Table 7. Independent sample t-test for two groups of creativity tendency consensus assessment.

The full score of students' works is 100. The average score of expert consensus assessment is 85 in experimental group A (creative thinking teaching method) and 76 in control group B (traditional teaching method).

Six experts have given feedback on the learning outcomes of these two groups. Expert E1 said works from the experiment group were more complete and were finished quite better. Besides, Expert E2 said works from the experiment group were made of various materials and were of exhibition value. Importantly, Expert E4 said works from the experiment group were colorful and creative, making people full of imagination. Based on the feedback from experts, the control group attached more importance to details and skills, making works lack of creativity, said the expert E3. Besides, Expert E5 pointed out that works from the control group were made carefully and used a lot of skills taught by their teachers in the class. Furthermore, Expert E6 said both groups used methods of making lanterns taught by their teachers, but the experiment group was bolder and attracted my attention.

The works of the experimental group reflect the characteristics of bold colors, extended decoration, and dramatic emotional expression, echoing three important aspects of behavior. The expert evaluation results of the experimental group are better than the control group in creativity, aesthetic taste, and technical advantage. It can be found that the students who accept creative thinking teaching methods are better than those who accept traditional teaching methods in academic performance. Table 8 shows the classroom teaching situation of the experimental group and control group.

TABLE 8
www.frontiersin.org

Table 8. Classroom teaching situation of experimental group A and control group B.

The results of expert consensus assessment show that creative thinking teaching method can effectively improve students' ability to conceive and create material works, in which the average value of creativity is the most significant, followed by technical advantages and aesthetic attraction, which is similar to that of Lai (2006) and Shih (2013).

The hypotheses of H1b, h3a, H3b, and H3C are valid after being verified by statistical methods. Figure 3 shows the summary of the results of the hypothesis in this study.

FIGURE 3
www.frontiersin.org

Figure 3. A summary of hypotheses with significant differences.

Discussion

Creativity is necessary for inventive thinking in any domain (Ocetkiewicz, 2021). However, is underappreciated in many Technical Education. This study integrates creative thinking teaching method into aesthetic beauty and hairdressing technical education courses. Students are divided into experimental group A and control group B to carry out creative thinking teaching methods and traditional teaching methods, respectively and make a comparative study on learning outcomes. The results show that there is a significant difference between the teaching method of creative thinking and the reaction level of learning outcomes. In the experimental group, the creative thinking teaching method is popular with students. These findings echoed the findings of previous studies of response level (Chang, 2004; Hung, 2004). Creative thinking can be applied to various disciplines, which can improve students' learning satisfaction (Li et al., 2011; Hu et al., 2016; Huang et al., 2021), also can enhance students' participation in the class (Tu et al., 2018). The creative thinking teaching method is positively correlated with creativity, aesthetic sensibility, and technical advantage. Therefore, the creative thinking teaching methods can effectively improve the learning effect. Additionally, creative thinking teaching method should contribute to the shifting away from traditional teaching, which on the learners as the broadcaster of content, toward a model based on functional learning.

In terms of teaching methods, the satisfaction with the creative thinking teaching method in the experimental group was higher than that of the traditional teaching method in the control group. The application of the brainstorming method, free association method, and six W's review method in the creative thinking teaching method can obviously stimulate students' more ideas and creativity, and effectively enhance their learning motivation, these findings are consistent with previous studies (Chen, 1999; Tan et al., 2016; Harris and de Bruin, 2018). As pointed out by Rodríguez et al. (2019) students rated their satisfaction with the course highly. That is to say, the creative thinking teaching method can effectively promote students' thinking and creativity, and the use of creative thinking teaching can help improve students' learning willingness and learning outcomes.

Although there's no significant difference between the creative thinking teaching methods and the creative tendency of learning outcomes, the creative thinking teaching methods hardly improve students' creative tendency. Even so, the scores of the creative tendency of the experimental group which applies the creative thinking teaching methods are better than that of the control group which applies the traditional teaching methods (Kennedy, 2010; Robson and Jaaniste, 2010; Davies et al., 2013). Among them, the difference in risk-taking level is the largest, the second is challenging, then are the imagination and curiosity, the latter of which the smallest. As is seen above, higher vocational students lack curiosity and imagination. In the future course arrangement, we suggest arranging more activities aimed to cultivate and stimulate students' curiosity and imagination. All teachers need to understand that creativity is a skill that may be developed in students (Ocetkiewicz, 2021). Accordingly, teachers must design strategies to support creative thinking teaching, thus encouraging and developing students' creativity.

According to expert consensus, creative thinking teaching methods can effectively improve students' ability to conceive multimedia material creations. The average of creativity improves the most and the next are the technical advantages and aesthetic sensibility, which is consistent with previous research results (Lai, 2006; Shih, 2013; Perry and Karpova, 2017). Knowing that every student can develop their creativity, teachers must formulate strategies to support creative thinking teaching, thus encouraging and fostering students' creativity.

Conclusion

This study aims at exploring the effect of creative thinking teaching methods on multimedia material creation. It shows that it is feasible to introduce this method to beauty and hairdressing technical education. Creative thinking teaching methods can improve students learning motives, and inspire students' will to try different study methods, thus enhancing their creativity and learning outcomes. However, teachers should spend more time preparing for courses but feel happy about students' enthusiasm. To foster students' ESD, teachers should try more student-centered teaching methods to guide students' awareness and foster their creative thinking. The study, if taught through creative thinking teaching methods, is challenging for students, who will have a sense of achievement and be more willing to be involved. An increasing number of students believe that more creativity and creative ideas will be inspired when students consider how to solve problems, thus encouraging them to make more attempts and get a better learning outcome.

According to this research, creative teaching is a way that teachers inherit traditional teaching methods. With diversified teaching methods, it can inspire students' learning motivation and performance, increase their interest in teaching activities and encourage them to think to further foster their creative thinking. In the multimedia material creation course of beauty and hairdressing technical education, the creative thinking teaching methods can effectively improve students' learning outcomes, transform their thinking mode and break their thinking limits. As a result, students can put forward different ideas and answers, and make constant innovation, creating more creative and meaningful works. As society changes in the future, the traditional single thinking mode will not work, but creative thinking will contribute to students' sustainable learning. The creative thinking teaching method can make important contributions and is of significant value to teaching practice and academic research.

Research limitations

This study also has some limitations. Firstly, the subjects of the study are 43 college students, which are limited in statistics. If the number of experimental samples can be increased, the inference statistics will be more accurate; Secondly, the gender distribution of the sample of research objects is mainly female (41 female students, accounting for 95%, 2 male students, accounting for 5%). The main reason is that the students of the department of applied cosmetology are mostly female, and the gender sample is very uneven, which makes the research results unable to verify whether the creative thinking teaching method is affected by gender. Secondly, Kirkpatrick proposed the four level training evaluations model (Kirkpatrick, 1975; Kirkpatrick and Kirkpatrick, 2006), because the “result level” is difficult to measure the time of learning outcomes, and it is not easy to determine the criteria for evaluating the results. Therefore, it is not included in the scope of the study. Future studies can include the “achievement level” to measure more accurate learning outcomes. Thirdly, due to the limitation of time, the research is limited to the differences in students' learning outcomes after the experimental course and fails to analyze the differences of students before and after the experimental course. Therefore, future research can further study the comparison between pre-test and post-test of experimental courses; Finally, if future researchers want to take college students as the research object, it is suggested to extend or increase the number of times in teaching time, expand the scope of teaching to other courses, and apply creative thinking teaching method more comprehensively, so as to promote students' creativity, so as to analyze whether students' creativity changes due to different teaching methods.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding authors.

Ethics statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author contributions

X-ZL and XK contributed to the ideas of creative thinking teaching research, collection of data, empirical analysis, the data analysis, design of research methods, and tables. C-CC and X-ZL participated in developing a research design, writing, and interpreting the analysis. X-ZL, C-CC, and XK contributed to the literature review and conclusions. All authors contributed to the article and approved the submitted version.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Al-Dababneh, K. A., Al-Zboon, E. K., and Ahmad, J. (2019). The creative environment: teachers' perceptions, self-efficacy, and teaching experience for fostering children's creativity. Early Child Dev. Care 189, 1620–1637. doi: 10.1080/03004430.2017.1400969

CrossRef Full Text | Google Scholar

Amabile, T. M. (1983). The social psychology of creativity: a componential conceptualization. J. Pers. Soc. Psychol. 45, 357. doi: 10.1037/0022-3514.45.2.357

CrossRef Full Text | Google Scholar

Amabile, T. M., Collins, M. A., Conti, R., Phillips, E., Picariello, M., Ruscio, J., et al. (2018). Creativity in Context: Update to the Social Psychology of Creativity. New York, NY: Routledge. doi: 10.4324/9780429501234

CrossRef Full Text | Google Scholar

Beghetto, R. A., and Kaufman, J. C. (2014). Classroom contexts for creativity. High Abil. Stud. 25, 53–69. doi: 10.1080/13598139.2014.905247

CrossRef Full Text | Google Scholar

Black, C., Freeman, C., and Stumpo, G. (2015). Conceptual model and strategies for creative thinking in apparel design. Int. J. Fashion Design Technol. Educ. 8, 131–138. doi: 10.1080/17543266.2015.1018958

CrossRef Full Text | Google Scholar

Brown, F. G. (1981). Measuring Classroom Achievement. Holt: Rinehart and Winston.

Burnard, P. (2012). Musical Creativities in Practice. Oxford University Press. doi: 10.1093/acprof:oso/9780199583942.001.0001

CrossRef Full Text | Google Scholar

Chan, C.-H. (2004). The Influcence of Creative Thinking Teaching on the Learning Effectiveness of the Technology Home Economies Student at Senior High School. Master's Thesis. National Taiwan Normal University. Taipei City. Available online at: https://hdl.handle.net/11296/ccg42f

Chang, C.-Y. (2004). A Study of the Effects of “Creativity Teaching into the Integrative Activities Learning Area” on Creative Performance for Junior High School Students. Master's Thesis. National Taiwan Normal University. Taipei City. Available online at: https://hdl.handle.net/11296/jkp94n

Chang, S.-H. (2013). Creativity: Theory, Techniques, and Teaching. Taipei City: Wu-Nan Book Inc.

Chen, C.-Y. (2003). A Study of Teaching Effectiveness of Machinery Mechanics Course With Creative-Thinking Instructional Strategy at Mechanical Engineering of Vocational High School. Master's Thesis. National Changhua University of Education. Changhua County. Available online at: https://hdl.handle.net/11296/uhfw8f

Chen, L.-A. (2006). The Theory and Practice of Creative Thinking Teaching. New Taipei City: The Profile of Psychological Publishing Co., Ltd.

Chen, L.-W. and Liu, W.-T. (2019). Analysis on the current development of education in Taiwan and South Korea's beauty technology. Taiwan Educ. Rev. Monthly 8, 136–161.

Chen, M. T. (1999). The Implementation Effect of the Creative Thinking Teaching Plan of the Hairdressing Technology Course in the Practical Skills Class of the Higher Vocational School. Master's Thesis. National Taiwan Normal University. Taipei City. Available online at: https://hdl.handle.net/11296/43d834

Chen, S.-Y., Lai, C.-F., Lai, Y.-H., and Su, Y.-S. (2022). Effect of project-based learning on development of students' creative thinking. Int. J. Elec. Eng. Educ. 59, 232–250. doi: 10.1177/0020720919846808

CrossRef Full Text | Google Scholar

Chou, C.-H. (2013). A Study on Effectiveness of Creativity Training to Promote Creativity on The College Students. Shih Chien University. Taipei City. Available online at: https://hdl.handle.net/11296/2hvwhr

Google Scholar

Chou, W.-S. (2004). A Study on Application of the Creative Problem Solving Teaching Method for the Gifted Junior High School Students. National Taiwan Normal University. Available online at: https://hdl.handle.net/11296/xasewd

Google Scholar

Cropley, A. J. (1997). Fostering creativity in the classroom: General principles. The Creativity Research Handbook 1, 1–46.

Google Scholar

Davies, D., Jindal-Snape, D., Collier, C., Digby, R., Hay, P., and Howe, A. (2013). Creative learning environments in education—a systematic literature review. Think. Skills Creat. 8, 80–91. doi: 10.1016/j.tsc.2012.07.004

CrossRef Full Text | Google Scholar

Dora To, L.-W., Phoebe Chan, Y.-L., Lam, Y. K., and Tsang, S.-K. Y. (2011). Reflections on a primary school teacher professional development programme on learning English through process drama. Res. Drama Educ.: J. Appl. Theatre Perform. 16, 517–539. doi: 10.1080/13569783.2011.617099

CrossRef Full Text | Google Scholar

Fadel, C., and Trilling, B. (2010). 21st Century Skills: Learning for Life in Our Times. Education Review.

Google Scholar

Gagné, R. M. (1985). Conditions of Learning and Theory of Instruction. Holt: Rinehart and Winston.

Google Scholar

Guilford, J. P. (1968). Intelligence, Creativity and Their Educational Implications. San Diego, CA: Robert R. Knapp.

Google Scholar

Harris, A., and de Bruin, L. (2018). An international study of creative pedagogies in practice in secondary schools: toward a creative ecology. J. Curric. Pedagogy 15, 215–235. doi: 10.1080/15505170.2018.1457999

CrossRef Full Text | Google Scholar

Hickey, M., and Webster, P. (2001). Creative thinking in music. Music Educ. J. 88, 19–23. doi: 10.2307/3399772

CrossRef Full Text | Google Scholar

Hsia, L.-H., Lin, Y.-N., and Hwang, G.-J. (2021). A creative problem solving-based flipped learning strategy for promoting students' performing creativity, skills and tendencies of creative thinking and collaboration. Br. J. Educ. Technol. 52, 1771–1787. doi: 10.1111/bjet.13073

CrossRef Full Text | Google Scholar

Hu, R., Wu, Y.-Y., and Shieh, C.-J. (2016). Effects of virtual reality integrated creative thinking instruction on students' creative thinking abilities. Eurasia J. Math. Sci. Technol. Educ. 12, 477–486. doi: 10.12973/eurasia.2016.1226a

CrossRef Full Text | Google Scholar

Huang, F.-C. (2011). The Study of the Effectiveness of Combining Creative Thinking with Hairdressing Course in Vocational Cosmetology Department. Tainan University of Technology. Tainan City. Available online at: https://hdl.handle.net/11296/qwtfj7

Google Scholar

Huang, S.-Y., Ko, P.-J., Lin, H.-H., Dai, R.-H., and Chen, H.-C. (2021). Creative thinking counseling teaching program can improve the creativity, creative tendency, and self-concept of Grade 7 students: a quasi-experimental study. J. Creat. Behav. 55, 819–838. doi: 10.1002/jocb.491

CrossRef Full Text | Google Scholar

Hung, C.-Y. (2011). The Study of Creativity Development—the Effects of Art Education, Innovative Teaching Behaviors and Parenting Styles on the Elementary School Students. Chung Hua University. Hsinchu City. Available online at: https://hdl.handle.net/11296/p4h666

Google Scholar

Hung, S.-M. (2004). A Study of Teaching Effectiveness of Visual Arts Curriculum with Creative Thinking Instructional Strategy In Elementary Schools. National Kaohsiung Normal University. Kaohsiung City. Available online at: https://hdl.handle.net/11296/y4vgu9

Google Scholar

Jeffries, K. K., Zamenopoulos, T., and Green, A. J. K. (2018). Design creativity, technical execution and aesthetic appeal: a CAT with caveats (Part 2). Int. J. Design Creat. Innov. 6, 66–79. doi: 10.1080/21650349.2017.1381043

CrossRef Full Text | Google Scholar

Jones, L. (2007). The Student-Centered Classroom. New York, NY: Cambridge University Press.

Google Scholar

Kennedy, A. (2010). Researching children researching the world: 5 × 5 × 5=creativity. Early Years 30, 95–96. doi: 10.1080/09575140903358133

CrossRef Full Text | Google Scholar

Kirkpatrick, D., and Kirkpatrick, J. (2006). Evaluating training programs: the four levels. J. Appl. Manage. Entrep. 11, 96–98.

Google Scholar

Kirkpatrick, D. L. (1975). Evaluating Training Programs. Tata McGraw-hill education.

Google Scholar

Kladder, J., and Lee, W. (2019). Music teachers perceptions of creativity: a preliminary investigation. Creat. Res. J. 31, 395–407. doi: 10.1080/10400419.2019.1651189

CrossRef Full Text | Google Scholar

Kousoulas, F. (2010). The interplay of creative behavior, divergent thinking, and knowledge base in students' creative expression during learning activity. Creat. Res. J. 22, 387–396. doi: 10.1080/10400419.2010.523404

CrossRef Full Text | Google Scholar

Kraft, R. (1978). Bike Riding and the Art of Learning. Change: The Magazine of Higher Learning.

Google Scholar

Ku, Y.-L., and Kuo, C.-L. (2016). Develop a framework of creative thinking teaching mode for RN-BSN students on the basis of the creative process of clinical nurses in Taiwan. Innov. Educ. Teach. Int. 53, 424–434. doi: 10.1080/14703297.2014.949282

CrossRef Full Text | Google Scholar

Lai, C.-C. (2006). The Confirmation of Amabile's Component Model of Creativity and Consensual Assessment Technique: The Evidence of Children's Drawing. National Sun Yat-sen University. Kaohsiung City. Available online at: https://hdl.handle.net/11296/4p263a

Google Scholar

Leicht, A., Heiss, J., and Byun, W. J. (2018). Issues and Trends in Education for Sustainable Development (vol. 5). Paris: UNESCO publishing.

Google Scholar

Li, K.-Y., Hau, C.-S., Huang, Y.-C., and Huang, D.-F. (2011). “A study of the influence of creative thinking instruction implemented in the engineering education “mold production practice” curriculum on the creativity of vocational high school students,” in Information and Management Engineering Berlin, ed M. Zhu (Springer: Heidelberg), 239–245. doi: 10.1007/978-3-642-24022-5_40

CrossRef Full Text | Google Scholar

Lien, B. Y. H., Hung, R. Y. Y., and McLean, G. N. (2007). Training evaluation based on cases of Taiwanese benchmarked high-tech companies. Int. J. Train. Dev. 11, 35–48. doi: 10.1111/j.1468-2419.2007.00268.x

CrossRef Full Text | Google Scholar

Lin, H.-S. (2008). A Study of the Effect on Visual Arts Creativity and Thinking Programfor the Art Class' Students in Junior High School. National Changhua University of Education. Changhua County. Available online at: https://hdl.handle.net/11296/745m68

Google Scholar

Lin, H.-T. (1974). The effects of creative teaching on the development of creativity of the gifted students. Bull. Educ. Res. 16, 321–382.

Google Scholar

Lin, H.-T., and Wang, M.-J. (1994). Creativity Assessment Packet. New Taipei City: The Profile of Psychological Publishing Co., Ltd.

Google Scholar

Liu, H.-Y., Wang, I.-T., Huang, D.-H., Hsu, D.-Y., and Han, H.-M. (2020). Nurturing and enhancing creativity of nursing students in Taiwan: a quasi-experimental study. J. Creat. Behav. 54, 799–814. doi: 10.1002/jocb.407

CrossRef Full Text | Google Scholar

Liu, X., Ni, C., Yang, R., Li, X., and Cheng, C. (2007). Teaching design of cultivating nursing students' creative thinking. Online Submission 4, 38–41.

PubMed Abstract | Google Scholar

Mitchell, R., and Martin, C. (1997). Rote learning, creativity and 'understanding' in classroom foreign language teaching. Lang. Teach. Res. 1, 1–27. doi: 10.1177/136216889700100102

CrossRef Full Text | Google Scholar

Molderez, I., and Ceulemans, K. (2018). The power of art to foster systems thinking, one of the key competencies of education for sustainable development. J. Clean. Prod. 186, 758–770. doi: 10.1016/j.jclepro.2018.03.120

CrossRef Full Text | Google Scholar

Mollahoseini, A., and Farjad, S. (2012). Assessment effectiveness on the job training in higher education (case study: Takestan University). Procedia Soc. Behav. Sci. 47, 1310–1314. doi: 10.1016/j.sbspro.2012.06.817

CrossRef Full Text | Google Scholar

Morris, W. (2006). Creativity its Place in Education. New Zealand: Future Edge Ltd.

Google Scholar

Ocetkiewicz, I. (2021). Creativity for sustainability: how do polish teachers develop students' creativity competence? Analysis of Research Results. Sustainability 13, 571. doi: 10.3390/su13020571

CrossRef Full Text | Google Scholar

Perry, A., and Karpova, E. (2017). Efficacy of teaching creative thinking skills: a comparison of multiple creativity assessments. Think. Skills Creat. 24, 118–126. doi: 10.1016/j.tsc.2017.02.017

CrossRef Full Text | Google Scholar

Rieckmann, M. (2017). Education for Sustainable Development Goals: Learning Objectives. Paris: Unesco Publishing.

Google Scholar

Robson, J., and Jaaniste, L. (2010). Growing Future Innovators: A New Approach To Learning Programs For Young People [Report]. Perth Institute of Contemporary Arts. Available online at: https://apo.org.au/node/23501

Google Scholar

Rodríguez, G., Pérez, N., Núñez, G., Baños, J.-E., and Carrió, M. (2019). Developing creative and research skills through an open and interprofessional inquiry-based learning course. BMC Med. Educ. 19, 134. doi: 10.1186/s12909-019-1563-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Schweisfurth, M., and Elliott, J. (2019). When ‘best practice' meets the pedagogical nexus: recontextualisation, reframing and resilience. Comp. Educ. 55, 1–8. doi: 10.1080/03050068.2018.1544801

CrossRef Full Text | Google Scholar

Shi, B., Lu, Y., Dai, D. Y., and Lin, C. (2013). Relationships between migration to urban settings and children's creative inclinations. Creat. Res. J. 25, 300–311. doi: 10.1080/10400419.2013.813793

CrossRef Full Text | Google Scholar

Shih, F.-T. (2013). The Influence of Creative Thinking Teaching in Arts and Life Class for the Students in Extensive Education Division-From Creative Advertising Curriculum. National Taiwan Normal University. Taipei City. Available online at: https://hdl.handle.net/11296/ct2hk5

Google Scholar

Sidiropoulos, E. (2022). The influence of higher education on student learning and agency for sustainability transition. Sustainability 14, 3098. doi: 10.3390/su14053098

CrossRef Full Text | Google Scholar

Su, Y.-S., Ding, T.-J., and Lai, C.-F. (2017). Analysis of students engagement and learning performance in a social community supported computer programming course. Eurasia J. Math. Sci. Technol. Educ. 13, 6189–6201. doi: 10.12973/eurasia.2017.01058a

CrossRef Full Text | Google Scholar

Tan, L. S., Lee, S. S., Ponnusamy, L. D., Koh, E. R., and Tan, K. C. K. (2016). Fostering creativity in the classroom for high ability students: context does matter. Educ. Sci. 6, 36. doi: 10.3390/educsci6040036

CrossRef Full Text | Google Scholar

Thomas, K., and Chan, J. (2013). Handbook of Research On Creativity. Cheltenham: Edward Elgar Publishing. doi: 10.4337/9780857939814

CrossRef Full Text | Google Scholar

Torrance, E. P. (1972). Torrance Tests of Creative Thinking—Directions Manual and Scoring Guide—Figural Test. Lexington: Personnel Pres. Inc.

Google Scholar

Tseng, C.-H. (2009). A Study of Creative Thinking Teaching Effectiveness on the Hairdressing Curriculum in Vocational High School. Chinese Culture University. Taipei City. Available online at: https://hdl.handle.net/11296/e2gyb9

Google Scholar

Tu, J.-C., Liu, L.-X., and Wu, K.-Y. (2018). Study on the learning effectiveness of stanford design thinking in integrated design education. Sustainability 10, 2649. doi: 10.3390/su10082649

CrossRef Full Text | Google Scholar

Wager, W. (2003). Writing learning outcomes-communicating expectations [writing learning outcomes-communicating expectations]. J. Gen. Educ. 10, 101–114.

Google Scholar

Wei, H.-T. (2001). A Study of the Effects of Creative Problem Solving Teaching on Technology Education in Junior High Schools. National Taiwan Normal University. Taipei City. Available online at: https://hdl.handle.net/11296/8r8w97

Google Scholar

Wu, P. L., Chiang, C. S., and Chen, H. S. (2014). The influence of beauty modeling training course in teaching efficiency – a case of creative beauty salon design in continuing educational University. Appl. Mech. Mat. 536, 1688–1696. doi: 10.4028/www.scientific.net/AMM.536-537.1688

CrossRef Full Text | Google Scholar

Wu, S. Q. (2002). The Influence of Creative Thinking Teaching on the Student Learning Effectiveness of the Living Technology Curriculums at Junior High School National. Taiwan Normal University. Taipei City. Available online at: https://hdl.handle.net/11296/w8s733

Wu, X., Zhang, L. J., and Liu, Q. (2021b). Using assessment for learning: multi-case studies of three Chinese University English as a foreign language (EFL) teachers engaging students in learning and assessment [original research]. Front. Psychol. 12, 4033. doi: 10.3389/fpsyg.2021.725132

PubMed Abstract | CrossRef Full Text | Google Scholar

Wu, X. M., Dixon, H. R., and Zhang, L. J. (2021a). Sustainable development of students' learning capabilities: the case of University students' attitudes towards teachers, peers, and themselves as oral feedback sources in learning english. Sustainability 13, 5211. doi: 10.3390/su13095211

CrossRef Full Text | Google Scholar

Zhou, R., and Lee, N. (2022). The reception of education for sustainable development (ESD) in China: a historical review. Sustainability 14, 4333. doi: 10.3390/su14074333

CrossRef Full Text | Google Scholar

Keywords: creative thinking teaching, learning outcomes, material creation course, vocational education, learning assessment

Citation: Li X-Z, Chen C-C and Kang X (2022) Research on the cultivation of sustainable development ability of higher vocational students by creative thinking teaching method. Front. Psychol. 13:979913. doi: 10.3389/fpsyg.2022.979913

Received: 28 June 2022; Accepted: 26 August 2022;
Published: 05 October 2022.

Edited by:

Jian-Hong Ye, Beijing Normal University, China

Reviewed by:

Kang Zhang, The University of Texas at Dallas, United States
Chulanee Thianthai, Chulalongkorn University, Thailand

Copyright © 2022 Li, Chen and Kang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Chun-Ching Chen, cceugene@ntut.edu.tw

These authors have contributed equally to this work and share first authorship

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.