Jen Katz-Buonincontro
Ross C. Anderson- 1School of Education, Drexel University, Philadelphia, PA, United States
- 2Department of Educational Methodology Policy and Leadership, University of Oregon, Eugene, OR, United States
Without quality research using observation methods, it is difficult to understand creativity processes in action, especially in the field of education. But, what is the current state of these studies? Based on a review of 37 extant studies, we found that observation is surprisingly under-utilized, and that more rigorous observation studies of creativity processes in education are needed.
Firsthand accounts of behaviors, interactions, and discussions between individuals or groups (Merriam and Tisdell, 2016) is an essential affordance of observation. In education, observation of student creativity focuses mostly on talented and gifted student identification (Plucker and Makel, 2010), with lesser attention to the classroom context and interactions among teachers and students. As most creativity studies are informed by psychological research methods and thus favor quantitative methods, often in the tradition of psychometrics and experimental design (e.g., Plucker and Renzulli, 1999; Plucker and Makel, 2010), observation methods have received relatively less attention. In this opinion article, we review 37 published studies using quantitative behavioral observation or qualitative naturalistic observation to study creativity in education, across 38 years. Our aim was to parse out the strengths and weaknesses of the articles and recommend improvements for the rigor of observation studies (see Table 1).
Table 1. Recommendations to address methodological weaknesses in observation of creativity in education.
Contributions of Quantitative Behavioral Observation
In quantitative observation methods, the researcher records evidence of individual behaviors or interactions (Creswell, 2008), but this method is very limited in educational creativity studies. In this review, we used search terms “observation/observational methods/or observational learning,” combined with “education,” and “creativ*,” “think,*” “problem solv*,” or “behav*” in ERIC, Education Research Complete, Art Education and PsycInfo databases, and specific journal archives. Nine articles were identified from 1980 to 2018 using quantitative observation methods for studying the learning process or environments related to creativity. Seven measures focused on environmental or instructional support for creativity in K-12 educational settings—five domain-general settings (5), one in K-12 science classrooms (1), and one in a physical education class (1). Two additional studies examined the creative process in different types of tasks (see Table 1).
In these articles, observational methods tested theories and evaluated educators. For example, Ruscio et al. (1998) recorded and measured aspects of the creative process of individuals in written, constructive, and artistic tasks. Torrents et al. (2010) used observation to measure creative movement in improvised physical interaction between two people. Schacter et al. (2006) investigated the relationship between observed teaching practices for creativity and students' subsequent academic achievement. One of the five studies published between 2016 and 2018 used observation to explore which aspects of support for creativity were most frequent in science instruction (Al-Abdali and Al-Balushi, 2016). Richardson and Mishra (2018) built their measure to help educators and school administrators improve the design of learning environments to support creativity. Konstantinidou and Zisi (2017) produced an 18-item checklist to evaluate the level of support for creativity in physical education classes. Gadja et al. (2017) observed both teachers and student behaviors separately using a checklist of exemplary creative behaviors in teaching and learning.
Creativity Framework
Most studies developed a framework of creative teaching behaviors through reviews of research on creativity and instructional practices within their domain of interest (Al-Abdali and Al-Balushi, 2016; Konstantinidou and Zisi, 2017; Richardson and Mishra, 2018). In some cases, researchers compared findings from past research with inductive findings from their own field notes in specific settings. Konstantinidou and Zisi (2017) adapted an extant self-report measure aligned to Cropley's (1997) framework to create a short behavioral observation checklist of the environment. Similarly, Furman (1998) applied the Origin-Pawn Questionnaire as a criterion for the cognitive, social, and emotional support for creativity. Al-Abdali and Al-Balushi (2016) framed their approach specifically around different science teaching approaches that support creativity. In exploratory studies, creativity theory guided a process that was largely inductive (Ruscio et al., 1998). Pitts et al. (2018) constructed an observation tool from a developmental framework for creativity in learning. Gadja et al. (2017) built their checklist of behaviors from an extant model of creativity in education.
The strongest quantitative observation approaches linked creativity theory explicitly to frameworks for instructional practices (Al-Abdali and Al-Balushi, 2016; Pitts et al., 2018; Richardson and Mishra, 2018). For example, Richardson and Mishra (2018) developed observation categories based on an extant observation protocol of school administrators, field notes in peer-nominated creative classrooms, and theory from creativity research.
Quantitative Data Collection and Analysis Methods
Schacter et al. (2006) observed 48 elementary school teachers eight different times for a whole class period using ethnographic field notes as the basis for ratings. Some studies provided extensive detail about the procedures used to observe and rate the environment (Schacter et al., 2006; Konstantinidou and Zisi, 2017; Richardson and Mishra, 2018) and others provided much less specificity (Ruscio et al., 1998; Al-Abdali and Al-Balushi, 2016). Generally, sampling decisions were based on convenience rather than analytic power.
Scoring procedures dictated the analytic approach for the observations. Across protocol researchers seemed to keep the number of indicators to score below 20. Most studies used a general response options for rating (e.g., “no evidence” to “fully present”); only Torrents et al. (2010) and Konstantinidou and Zisi (2017) used a behavioral checklist with actual frequency counts. Most studies included more than one observation instance and each study pursued inter-coder reliability, generally using the 80% threshold of inter-observer agreement across raters. In their analyses, most studies reported the descriptive statistics for each category or domain of indicators. Some researchers explored what different levels of frequency or quality might indicate for theory and practice (Schacter et al., 2006; Torrents et al., 2010; Al-Abdali and Al-Balushi, 2016; Konstantinidou and Zisi, 2017). Gadja et al. (2017) compared observation scores between classrooms identified as having a null, positive, or negative relationship between students' creative ability and academic achievement.
Each study established initial validity and usability of the instrument to observe for support for creativity. Like Al-Abdali and Al-Balushi (2016), other studies used a panel of judges germane to the context (e.g., science educators) to evaluate the face validity of the protocol and improve its practical significance. These quantitative studies represent early, exploratory work to promulgate research about the creative process and to improve the affordances of learning environments in support of creativity.
Contributions of Qualitative Observation Research
Qualitative observation methods were more prevalent than quantitative and mixed methods for observing creativity in learning environments. Twenty-six articles were found in ERIC and PsycInfo databases from 1980 to 2018 using qualitative observation methods for studying creativity processes in visual art (14), music education (3), theater (1), fashion (1), science, computer science and mathematics education (5), early childhood/non-domain specific (1), and technology (1).
Creativity Framework
Definitions of creativity ranged, and some authors elected to include multiple definitions (e.g., Pitri, 2013), making it difficult to pinpoint the study's contributions to specific creativity theories. Cognitive components of divergent and convergent thinking—fluency and flexibility—were emphasized (e.g., Karademir, 2016). Robson and Rowe (2012) used a creative thinking framework that conflated “imagination” with “creativity,” explaining it as originality, novelty, and even critical thinking. In arts and music education studies, the creativity focus shifted from cognitive components to more expressive qualities such as collaboration (Biasutti, 2015), modeling and employment of art materials (James, 1997; Guay, 2000; Kandemir and Gur, 2007; Geist and Hohn, 2009; Thomas, 2009; Walker, 2014; Budge, 2016; Lorimer, 2016; Mars, 2016), creative “encounters” (Petsch, 2000), aesthetic or flow experience (Vuk et al., 2015), and transformation (Walker, 2014).
While some articles described the research focus, such as teaching method (Walker, 2014), professional development (Lorimer, 2016), and student art activities (Thomas, 2009; Pitri, 2013), these articles were not connected to an intentional research design. Research designs included case study (James, 1997; Karademir, 2016) and ethnography (Guay, 2000; Petsch, 2000; Mars, 2016). Some studies attempted to tie general pedagogy to teaching for creativity in specific subjects. For example, Sullivan (2011) and Meyer and Lederman (2013) mapped science instruction to the demonstration of creativity, and Donovan et al. (2014) mapped creativity to technology. Conversely, other studies used an inductive framework. Robson and Rowe (2012) first identified child-initiated activities, which were then interpreted as supportive of creative thinking (p. 355). Walmsley (2013) focused on observing a co-creation process using theater, characterizing by “creative energy” and “rawness” (p. 114).
Data Collection and Analysis Methods
Several articles omitted citations on the process for collecting observation data. As a result, this obscured the researchers' process for data saturation to ensure a thorough understanding of the creative process. One article used critical incident sampling by selecting members for classroom observation based on reporting in surveys or interviews (Meyer and Lederman, 2013). Other studies observed students identified as gifted (e.g., Karademir, 2016).
Researchers used “thick description” (Geertz, 1973) to provide detailed description of the observation site and activities and thus demonstrate authenticity and validity of data. Some articles also used triangulation—the integration of observation and interview data to corroborate themes and establish validity across data sources (Waite, 2014). Thick description of the creative process included examples of modeling in art instruction (Budge, 2016), online music collaboration (Biasutti, 2015) and student-teacher interactions (James, 1997; Thomas, 2009). Photographs of student drawings and creative projects (Pitri, 2013; Walker, 2014; Vuk et al., 2015; Karademir, 2016; Lorimer, 2016) also strengthened observation descriptions.
Qualitative arts-based researchers build relationships with artists and students when observing the artistic process (Bresler, 2008). Accordingly, researchers discuss how empathy can deepen understanding and counteract bias (Bogdan and Biklen, 2007) to enhance validity. Most studies, though, did not disclose the type of observer role selected (e.g., Walker, 2014); a few specified the participant type (Guay, 2000; Petsch, 2000; Budge, 2016; Mars, 2016), and others described researcher involvement to understand student work in the classroom (James, 1997; Thomas, 2009; Pitri, 2013; Donovan et al., 2014; Lorimer, 2016).
Data analysis procedures varied, often using qualitative observations to triangulate—that is, confirm or disconfirm other data sources. In some articles, interview and survey results overshadowed observation data (Bertling, 2015; Lorimer, 2016) making it difficult to parse out the contributions that observation data made to understanding the nature of creativity (Godart and Mears, 2009; Watson, 2014). Many analytic coding procedures were not supported by methodological citations. Even when specified, the preparation of field notes for analysis and the relation of discrete codes to final themes remained vague. Examples of well-articulated coding of observations included the constant comparative method (Meyer and Lederman, 2013), content analysis method (Karademir, 2016), interaction analysis (Sullivan, 2011), semantic analysis (Thomas, 2009) and symbolic interactionism (James, 1997).
Conclusion
This review shows that observation methods are woefully underutilized and, generally, lack methodological rigor. It's imperative for researchers to uphold rigorous research standards in the area of observational studies of creativity in learning, just as they would when using creativity measures, for example. The varied quality of observation studies in this review highlights the need for clearer and stronger observation methods to advance the understanding of creativity processes in learning.
The lack of definitional clarity emphasized by Plucker et al. (2004) more than a decade ago remains a critical need in observation studies. Researchers in both qualitative and quantitative traditions must clarify how they define, operationalize, and observe creativity. We found that the majority of observation studies lacked conceptual specificity, short-changing their potential contributions to the field. Students should be encouraged to use observation of teaching and learning creativity as an empirical method in research courses and dissertation work, but they must be equipped with the methodological skill and conceptual rigor to link creative development to specific instructional and environmental factors. Strong student research will help build a pipeline of new researchers observing creativity in a broader variety of domains.
The dominance of product-only assessments of creativity in research explains in part why the creativity field struggles to develop and scale curricular and instructional supports that promote the creative process in everyday teaching and learning in Cho et al. (2013). As we have described, several diverse exemplars in the creativity literature demonstrate how rigorous observation studies can help produce richer and clearer accounts of creativity in educational settings. Without continued investment, observation in creativity research may remain an underdeveloped method. Furthermore, instructional and environmental progress in support of creative development of students will lag behind our ambitions as a field.
Author Contributions
JK-B developed the concept of the review, searched for the articles and reviewed the articles, wrote the intro, conclusion and the qualitative review. RA helped develop the concept and reviewed the quantitative articles and wrote the quantitative review and added to the table.
Funding
This work was supported by the U.S. Department of Education under PR/Award number U351D140063.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Al-Abdali, N. S., and Al-Balushi, S. M. (2016). Teaching for creativity by science teachers in grades 5–10. Int. J. Sci. Math. Educ. 14, 251–268. doi: 10.1007/s10763-014-9612-3
Bertling, J. G. (2015). The art of empathy: a mixed methods case study of a critical place-based art education program. Int. J. Educ. Arts 16, 1–27. Available online at: https://files.eric.ed.gov/fulltext/EJ1074143.pdf
Biasutti, M. (2015). Creativity in virtual spaces: communication modes employed during collaborative online music composition. Think. Skills Creativ. 17, 117–129. doi: 10.1016/j.tsc.2015.06.002
Bogdan, R., and Biklen, S. K. (2007). Qualitative Research for Education: An Introduction to Theories and Methods, 5th Edn. Boston, MA: Allyn and Bacon.
Bresler, L. (2008). Research as experience and the experience of research: mutual shaping in the arts and in qualitative inquiry. Learn. Landsc. 2, 267–279.
Budge, K. (2016). Teaching art and design: communicating creative practice through embodied and tacit knowledge. Arts Hum. High. Educ. 15, 432–445. doi: 10.1177/1474022215592247
Cho, Y., Chung, H. Y., Choi, K., Seo, C., and Baek, E. (2013). The emergence of student creativity in classroom settings: a case study of elementary schools in Korea. J. Creat. Behav. 47, 152–169. doi: 10.1002/jocb.29
Creswell, J. W. (2008). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 2nd Edn. Thousand Oaks, CA: Sage.
Cropley, A. J. (1997). “Fostering creativity in the classroom: General principles,” in The Creativity Research Handbook, Vol. 1, ed M. Runco (Cresskill, NJ: Hampton Press), 83–114.
Donovan, L., Green, T. D., and Mason, C. (2014). Examining the 21st century classroom: developing an innovation configuration map. J. Educ. Comput. Res. 50, 161–178. doi: 10.2190/EC.50.2.a
Furman, A. (1998). Teacher and pupil characteristics in the perception of the creativity of classroom climate. J. Creat. Behav. 32, 258–277. doi: 10.1002/j.2162-6057.1998.tb00821.x
Gadja, A., Beghetto, R., and Karwowski, M. (2017). Exploring creative learning in the classroom: a multi-method approach. Think. Skills Creativ. 24, 250–267. doi: 10.1016/j.tsc.2017.04.002
Geertz, C. (1973). “Thick description: toward an interpretive theory of culture,” in The Interpretation of Culture, ed C. Geertz (New York, NY: Basic Books), 3–30.
Geist, E., and Hohn, J. (2009). Encouraging creativity in the face of administrative convenience: how our schools discourage divergent thinking. Education 130, 141–151.
Godart, F. C., and Mears, A. (2009). How do cultural producers make creative decisions? Lessons from the catwalk. Soc. Forc. 88, 671–692. doi: 10.1353/sof.0.0266
Guay, D. M. (2000). Values, beliefs, behaviors, and artmaking in the middle grades: a teaching story. Visual Arts Res. 26, 38–52.
James, P. (1997). Learning artistic creativity: a case study. Studies Art Educ. 39, 74–88. doi: 10.2307/1320720
Kandemir, M. A., and Gur, H. (2007). Creativity training in problem solving: a model of creativity in mathematics teacher education. N. Horizons Educ. 55, 107–122. Available online at: https://files.eric.ed.gov/fulltext/EJ832896.pdf
Karademir, E. (2016). Investigation the scientific creativity of gifted students through project- based activities. Int. J. Res. Educ. Sci. 2, 416–427. doi: 10.21890/ijres.05662
Konstantinidou, E. P., and Zisi, V. Z. (2017). Do physical educators promote students' creativity? an observational analysis study. Phys. Educ. 74:420. doi: 10.18666/TPE-2017-V74-I3-7407
Lorimer, M. R. (2016). Cultivating aesthetic and creative expression: an arts-based professional development project for migrant education. Art Educ. 69, 35–43. doi: 10.1080/00043125.2016.1158592
Mars, A. (2016). Past and present intertwining when learning is at stake: composing and learning in a music theatre project. Int. J. Educ. Arts 17, 1–28. Available online at: http://www.ijea.org/v17n23/
Merriam, S. B., and Tisdell, E. (2016). Qualitative Research: A Guide to Design and Implementation, 4th Edn. San Francisco, CA: Jossey-Bass.
Meyer, A. A., and Lederman, N. G. (2013). Inventing creativity: an exploration of the pedagogy of ingenuity in science classrooms. School Sci. Math. 113, 400–409. doi: 10.1111/ssm.12039
Petsch, J. (2000). Creative encounters at the interaction factory. Visual Arts Res. 26, 85–93. Available online at: https://www.jstor.org/stable/20716000?seq=1#page_scan_tab_contents
Pitri, E. (2013). Skills and dispositions for creative problem solving during the artmaking process. Art Educ. 66, 41–46. doi: 10.1080/00043125.2013.11519215
Pitts, C., Anderson, R. C., and Haney, M. (2018). Measure of Instruction for Creative Engagement: making metacognition, modeling, and creative thinking visible. Learn. Environ. Res. 21, 43–59. doi: 10.1007/s10984-017-9238-9
Plucker, J. A., Beghetto, R. A., and Dow, G. T. (2004). Why isn't creativity more important to educational psychologists? Potentials, pitfalls, and future directions in creativity research. Educ. Psychol. 39, 83–96. doi: 10.1207/s15326985ep39021
Plucker, J. A., and Makel, M. C. (2010). “Assessment of creativity,” in Handbook of Creativity, ed R. J. Sternberg (New York, NY: Cambridge University Press), 48–73
Plucker, J. A., and Renzulli, J. S. (1999). “Psychometric approaches to the study of human creativity,” in Handbook of Creativity, ed R. J. Sternberg (New York, NY: Cambridge University Press), 35–61.
Richardson, C., and Mishra, P. (2018). Learning environments that support student creativity: developing the SCALE. Think. Skills Creativ. 27, 45–54. doi: 10.1016/j.tsc.2017.11.004
Robson, S., and Rowe, V. (2012). Observing young children's creative thinking: engagement, involvement and persistence. Int. J. Early Years Educ. 20, 349–364. doi: 10.1080/09669760.2012.743098
Ruscio, J., Whitney, D. M., and Amabile, T. M. (1998). Looking inside the fishbowl of creativity: verbal and behavioral predictors of creative performance. Creat. Res. J. 11, 243–263. doi: 10.1207/s15326934crj11034
Schacter, J., Thum, Y. M., and Zifkin, D. (2006). How much does creative teaching enhance elementary school students' achievement? J. Creat. Behav. 40, 47–72. doi: 10.1002/j.2162-6057.2006.tb01266.x
Sullivan, F. R. (2011). Serious and playful inquiry: epistemological aspects of collaborative creativity. J. Educ. Technol. Soc. 14:55. Available online at: https://www.jstor.org/stable/jeductechsoci.14.1.55
Thomas, K. (2009). Creativity in artmaking as a function of misrecognition in teacher-student relations in the final year of schooling. Stud. Art Educ. 51, 64–76. doi: 10.1080/00393541.2009.11518791
Torrents, C., Castañer, M., Dinušov,á, M., and Anguera, M. T. (2010). Discovering new ways of moving: observational analysis of motor creativity while dancing contact improvisation and the influence of the partner. J. Creat. Behav. 44, 53–69. doi: 10.1002/j.2162-6057.2010.tb01325.x
Vuk, S., Tacol, T., and Vogrinc, J. (2015). Adoption of the creative process according to the immersive method. CEPS J. 5:51. Available online at: https://ojs.cepsj.si/index.php/cepsj/article/view/127
Waite, D. (2014). Teaching the unteachable: some issues of qualitative research pedagogy. Qual. Inquiry 20, 267–281. doi: 10.1177/1077800413489532
Walker, M. A. (2014). From theory to practice: concept-based inquiry in a high school art classroom. Stud. Art Educ. 55, 287–*299. doi: 10.1080/00393541.2014.11518938
Walmsley, B. (2013). Co-creating theatre: authentic engagement or inter-legitimation? Cult. Trends 22, 108–118. doi: 10.1080/09548963.2013.783176
Keywords: creativity processes, observation, literature review, research methods education, qualitative methodology, quantitative methodology
Citation: Katz-Buonincontro J and Anderson RC (2018) How Do We Get From Good to Great? The Need for Better Observation Studies of Creativity in Education. Front. Psychol. 9:2342. doi: 10.3389/fpsyg.2018.02342
Received: 23 April 2018; Accepted: 08 November 2018;
Published: 27 November 2018.
Edited by:
Roni Reiter-Palmon, University of Nebraska Omaha, United StatesReviewed by:
Jonathan Plucker, Johns Hopkins University, United StatesPaul T. Sowden, University of Winchester, United Kingdom
Copyright © 2018 Katz-Buonincontro and Anderson. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Jen Katz-Buonincontro, amtiQGRyZXhlbC5lZHU=