Ulrike Krein
Anna Hartenstein
Mandy Schiefner-Rohs- Department of Educational Sciences, University of Kaiserslautern, Kaiserslautern, Germany
Digital data have a major impact on school practices and play a central role for teachers, including their pedagogical practice. From a research perspective, the question arises how data practices and data-related transformation processes in schools can be studied. Therefore, the aim of this paper is to methodically review research approaches and underlying methodological assumptions about data practices in schools based on a systematic review. The focus is on social science research designs, social science research instruments, and knowledge production methods. The article provides an overview of previous research practice in this area and concludes with possible implications for future research.
Introduction
In a school increasingly permeated by digitalization, the generation and use of data is becoming more and more relevant (Jarke and Breiter, 2019). On the one hand, it is possible to actively generate more data to support and/or monitor school and learner performance. On the other hand, however, in many cases data generation occurs unconsciously and incidentally solely through the use of certain software and/or apps (e.g., Selwyn et al., 2015), because the use of digital technologies alone generates data and data traces in the background that can subsequently be evaluated and interpreted (Breiter and Hepp, 2018). In the school context, for instance, digital traces or metadata are collected through the use of learning platforms. This data can, for example, make registration times or the number of correctly completed tasks of individual learners visible to many target groups, such as teachers but also parents (ibid.). The more digital services a school uses, the more (meta-)data is generated and the more processes are mapped in systems and thus transferred into digital data. Whereas the digital class register “used to” be viewable on paper only by a specific group, a digital class register is no longer so easily filed away in a cupboard: Many people can view it, and more and more data, such as about absences, grades and incidents, converge in it, which can now be quickly correlated or prepared in descriptive charts. However, schools, as places of formal education, have always depended on “translating” teaching and learning into data–just think of the evaluation or grading of performance or the management of schools. In organizational contexts in particular, data were viewed more from the perspective of administrative oversight. They are used as an element of (a) school evaluation (especially in the United States or Netherlands, see Anagnostopoulos et al., 2013) or (b) school development. Here, they are used as an internal school management tool to link performance data, administrative and self-generated data (Schildkamp et al., 2012a), as is actively implemented in school performance studies, for example.
At the same time, the use of digital data under the terms “learning analytics” and “educational data mining” promises to reduce the burden on teachers and is often enthusiastically promoted as an innovative means to improve teaching and learning (cf. Selwyn et al., 2021, p. 2; Dander and Aßmann, 2015). Also linked to this are educational policy and pedagogical imperatives for school actors (Selwyn, 2020) to proactively use data for their own actions: “Central requirements here are the ability to ‘perceive’, ‘interpret’, and ‘construct implications’ from data [...]” (ibid., p. 2). Therefore, it is not surprising that algorithms, datafication, and data-based or even data-driven (Houben and Prietl, 2018) practices of educational actors in schools are receiving increasing (inter)national attention (Espeland and Stevens, 2008; Kitchin, 2016; Mau, 2017). However, although several research projects have addressed this issue, the particular role of data practices, understood as constituting elements of social life (see below; also Krein and Schiefner-Rohs., 2021), by school actors and pedagogical staff has been little explored. We therefore wondered how data and related data practices in schools have so far been addressed in research discourse internationally.
Theoretical background
If one wants to understand professional doing and its increasingly digitalizing relation to the world, it is important to consider media pedagogical practices. What is relevant in this context is that media-based pedagogical practices no longer exist merely as an interplay of active teachers, learners, and passive, artificially created things, but are grasped as an interwoven teacher-learner-thing interaction (cf. Allert and Richter, 2017). In school practices and teaching, digital media are not neutral objects or mere tools that function as purely didactic aids and obtain their function through their users in practice. Rather, logics and offers for action are inscribed in them, in the case of programs and algorithms even in the literal sense. These actively and often unpredictably participate in practices; at the same time, they condition practices, if one thinks of the associated process of datafication. For example, what teachers or students get to read in apps or on the web is primarily related to their respective ‘click histories’ and the data collected and evaluated about them.
Research on data practices in schools is therefore only just beginning to emerge, even at the definitional level, since there is no uniform understanding of data and data practices in the current discourse on school and media pedagogy (Krein and Schiefner-Rohs., 2021): Thus, research can be found that, on the one hand, understands data practices in the general sense of the word as actions, performances, and resulting consequences. This understanding implies the use of data-producing technologies in everyday educational life, i.e., everything that is “done” in, with, and through “data” in school–keyword “doing data” (e.g., Selwyn, 2015; Smith, 2018; Decuypere, 2021). On the other hand, data practices are defined in other research as constituting elements of social life (Decuypere, 2021). This broadens the perspective to include the various ways in which data are “brought to life,” how actors relate to and work with data in practice (Selwyn, 2020, p. 3). Accordingly, “data action” (ibid.) and consideration of how data practices mutually influence actors’ ways of thinking and acting becomes central (Selwyn and Pangrazio, 2018). Considering the discourse on digital data in schools with the theoretical approach of social practices (cf. Schatzki, 1996; Reckwitz, 2003), the interplay between data practices and sociomateriality also becomes clear. Since from this perspective, technologies are then no longer just “neutral tools [...] that do their work in the hands of teachers and learners without further ado” (Röhl, 2013, p. 2). On the contrary, in this understanding technologies are constitutive participants in practices (Röhl, 2013). The focus, then, is to examine how different actors connect and interact with technology to address practices. Digital media are then, according to Kalthoff et al. (2016), actors that enable and constrain practices. Moreover, the invisible elements of data, e.g., data infrastructures or algorithms, also become relevant. Conceptualizing data practices in this way has implications for understanding: first, that data practices are processual, and second, that data points emerge as outcomes of data practices, or that practice and data point coincide (Decuypere, 2021). This means that data practices become largely opaque or “invisible” from the outside. Inherent in this relational perspective, which underlies data practices in schools, are new challenges, especially in terms of concrete methodological approaches to study them (vgl. Gulson et al., 2017; van de Oudeweetering and Decuypere, 2021). For example, Decuypere (2021) currently notes a “lack of scholarship and corresponding methodological proposals that allow us to critically examine the concrete doings, impacts, and effects of data practices on the social field in general and the field of education in particular” (Decuypere, 2021, p. 68).
Following on from this, the question that arises from a research methodological perspective is how data practices in schools may be studied or have been explored to date. This paper aims to address this question and provide an overview of previous approaches to how data practices, especially in schools, have been methodologically conceptualized and methodologically investigated so far. Thus, the guiding research question is: How are data-based practices and transformational processes studied in schools? Specifically, the following questions were explored:
(1) What method(olog)ical research approaches have been used so far to examine data practices in schools?
(2) What was the focus or aim of the studies?
(3) Which disciplinary perspectives are apparent in previous studies on data practices in schools?
The aim of this paper is to review the current state of research in terms of method(olog)ical approaches. For this purpose, previous studies were systematically reviewed, research approaches were presented and critically discussed. In order to understand the approach taken in the paper, the methodological procedure is first described below (section 3), before the results are presented in detail (section 4) and finally synthesized and discussed together (section 5).
Methodology
For a comprehensive overview of previous methodological approaches on data-practices in schools, a systematic mapping was conducted according to Newman and Gough (2020). This systematic mapping is intended to provide a better “understanding of the breadth, purpose, and extent of research activity on a phenomenon” (Newman and Gough, 2020, p. 16). For this purpose, a two-step procedure consisting of a literature search and analysis of the obtained literature corpus was carried out. In the following, the underlying search strategy for the compilation of the data corpus is explained, before the coding and analysis of the data sets is presented.
Literature search strategy, databases and search terms
In a first step, a literature research was conducted in the second half of the year 2020 by using relevant databases such as FIS Education, SAGE Journals, ERIC, and SCOPUS to find articles dealing with data or datafication in schools. Defined search terms such as “data,” “datafication,” “practices,” and “school” linked with the Boolean operators OR and AND. The search yielded several 1,000 results, of which the abstracts were first examined for their fit to data-practice in the school context. After that, the literature was included or excluded according to the following criteria:
• The publications are scientific publications available in English or German.
• The publications were published between 2009 and 2019.
• The publications describe empirical (qualitative, quantitative, and mixed methods) studies; both peer-reviewed and non-peer-reviewed publications were included.
• The publications focus on schools and data-practices.
• The publications address pedagogical staff, teachers, and/or school leaders as target groups.
A total of 38 publications on the topic of data, datafication, and pedagogical practices in the school context were included in this review.
To fill remaining gaps in the literature review, additional searches of the reference lists of the selected datasets were conducted using the snowballing method (Wohlin, 2014). Literature recommendations were also obtained from international colleagues in the research field. This identified an additional 32 texts that met the above inclusion criteria and were thus included in the review. In total, the data corpus of the present article thus comprises N = 70 publications (Figure 1).
Figure 1. Identification of publications via databases and snowballing.
Thematic Coding and Analysis
An inductive, systematic coding method was selected to analyze the literature (see, e.g., Newman and Gough, 2020). To provide an overview of the state of research to date on data practices in schools, each article was analyzed in detail in terms of (1) geographic location and discipline, (2) research focus and intended audience, (3) research designs (data collection and analysis), and (4) methodological approaches. In this way, we were able to identify and trace highlights and trends in research on data practices in schools.
In the following, the individual results of the analyses are presented before these are taken up in the discussion and research-theoretical as well as research-practical implications are presented. The paper concludes with a reflection on the findings obtained and research gaps identified, as well as an outlook for future research.
Findings
This paper addresses the question of how data practices in schools have been researched and aims to provide insight into the methodological approaches used to date. The following section presents the results of the analyses described above. For this purpose, the geographical distribution as well as the disciplines found are first outlined, followed by an overview of the respective content foci and objectives of the analyzed papers. Subsequently, the research designs used in the literature will be presented and systematized before a mapping of the methodological frameworks used will be undertaken.
Geographical distribution and disciplines
In terms of the geographic distribution of articles, it is clear that due to the international nature of our analyses as well as the predefined languages of contribution, articles from all over the world were included in the study (see Figures 1, 2). The high number of publications from the United States (e.g., Honig and Venkateswaran, 2012; Spillane, 2012; Wayman et al., 2012; Jimerson and Reames, 2015; Jimerson, 2016; Sun et al., 2016; Park et al., 2017; Mandinach and Miskell, 2017a,b; Jimerson et al., 2018; Makkonen and Crane, 2019; Datnow et al., 2020) and Australia (Selwyn et al., 2015; Selwyn, 2020), which are often substantively concerned with educational policy and leadership, is striking in this regard, but so is a focus in the Netherlands (e.g., Schildkamp et al., 2012b, 2017; van der Kleij et al., 2015; Bolhuis et al., 2016; Brown et al., 2017; Hubers et al., 2018; Schildkamp, 2019; Decuypere, 2021). This shows, on the one hand, a long practice of data-driven education policy, which is then reflected in corresponding research projects, and, on the other hand, specific research foci of some researchers, e.g., Kim Schildkamp from the University of Twente in Netherlands.
Figure 2. Geographical distribution and focus of the publications found.
In addition, some contributions on data use in schools could be found from Germany (Förschler et al., 2021) and the United Kingdom (Gulson and Sellar, 2019). It should be noted, however, that in many cases these are not contributions with a genuine practice theory perspective (see Introduction), but rather contributions that focus predominantly on the use of data for evidence-based management of schools (e.g., Coburn and Talbert, 2006; Bowers, 2016; Gulson and Sellar, 2019). In addition, many of the articles we reviewed also address topics such as data-informed decision making (e.g., Honig and Coburn, 2008; Sun et al., 2016; Espin et al., 2017; Hubers et al., 2018; Schildkamp, 2019; Vanlommel and Schildkamp, 2019) and school evaluation (e.g., Schildkamp and Visscher, 2009; Schildkamp et al., 2009; Wayman et al., 2012). However, when looking at the authors, in addition to specific research foci of individuals, there is also a great diversity in the disciplines of researchers publishing in this area: There are studies from the fields of organizational research, policy & technology studies, human development, and digital education.
Content foci and aims
Furthermore, the individual articles included in the analysis revealed different areas of focus in terms of content, depending on the author(s). The focal points and goals explicitly mentioned in the respective literature can be summarized in the following five areas:
The first focus was on understanding data practices. In this context, there was an increased emphasis on “a deeper understanding of teachers’ individual sensemaking processes and reasoning in their specific context” (Vanlommel and Schildkamp, 2019, p. 10; also Park and Datnow, 2009; Park et al., 2012; Datnow et al., 2013). One focus in this context has been on understanding specific parts of data practice: for example, measurement and monitoring, as well as formative or summative assessment (e.g., Schildkamp and Kuiper, 2010; Selwyn, 2015). Striking here, moreover, is the decision-making aspect of data-based decisions (Datnow and Park, 2018, p. 148). Secondly, we found a focus on the factors influencing the use of data (Coburn and Turner, 2011; Schildkamp et al., 2017, 2019b; Prenger and Schildkamp, 2018). This, as Prenger and Schildkamp phrase it, aims “to explore which psychological factors contribute to teachers’ data use” (Prenger and Schildkamp, 2018, p. 734) and also tries to investigate the “factors influencing the use of data in data teams “(Schildkamp et al., 2017, p. 4). The third focus concentrates on effects of data use. In this context, the goal is on analyzing and/or evaluating what impact data use has or has had; for example: “to determine how social network structures reflecting knowledge sharing and brokerage changed over the first year in which schools started working with the data team intervention” (e.g., Hubers et al., 2018, p. 17). Likewise, a forth focus was on capacity development. Main goal here lies in enhancing educators’ data literacy skills, e.g., investigating the data team participants’ learning in depth (e.g., Bolhuis et al., 2016). One last focus we could identify is on perspectives on research in the topic area. Here, we subsumed studies that aim to contribute to research on the phenomenon by developing research tools (e.g., Jimerson, 2016). Unfortunately, these were few authors and papers.
Research designs and methodological frameworks
In considering the research designs reported in the papers, we took a multilevel approach: First, we categorized the research designs in terms of whether qualitative or quantitative methods were used. In a second step, we classified the studies according to whether they focused more on a leadership perspective – and thus school management, school development, or educational governance – or whether they dealt more with teaching and classroom management from the perspective of teachers. The wording of the respective methodological approach was taken from the studies. Not all of the studies could be clearly assigned to the respective categories, such as the study by Levin and Datnow (2012), which examines the perspectives of school leadership, schools, and teachers. Therefore, we decided to locate the methodological approaches along the spectrum of the 4-field matrix based on the categories we chose. In this regard, our categories form the extreme points of the following four axes, based on which the mapping took place. The methodological approaches are located first by research method and second by research focus according to their relationships to the categories (see Figure 3).
Figure 3. Research designs and methodological frameworks (reported) in the empirical literature.
It is striking that there are both quantitative and qualitative designs, as well as mixed-method designs that incorporate aspects of both. When looking at the designs in combination with the content focus of the study, different findings can be noted: Studies in our review that focus primarily concerned with a school perspectives and leadership issues tend to be more likely located in the quantitative paradigm (e.g., Schildkamp and Teddlie, 2007; Bowers, 2009; Schildkamp and Visscher, 2010a; Schildkamp et al., 2017; Prenger and Schildkamp, 2018; Makkonen and Crane, 2019; Ercan et al., 2021). Teacher-related or teaching-related perspectives, in contrast, are more likely to be addressed in qualitative designs (e.g., Jimerson and Reames, 2015; Datnow and Park, 2018; Vanlommel and Schildkamp, 2019). Besides the methodological approaches and paradigms, it is also noticeable that a large number of the studies examined have an explorative character or report their study design as explorative (e.g., Park et al., 2012; Bertrand and Marsh, 2015; Espin et al., 2017; Prenger and Schildkamp, 2018; Lasater et al., 2020). In this context, the number of reported case studies also predominates (e.g., Coburn and Talbert, 2006; Taveras et al., 2010; Park et al., 2012; Datnow et al., 2013; Bolhuis et al., 2016). These case studies were mainly qualitative research, using primarily traditional methods such as interviews (e.g., Schildkamp et al., 2012a; Vanlommel and Schildkamp, 2019), usually in combination with ethnographic methods such as site visits (e.g., Coburn and Talbert, 2006; Wohlstetter et al., 2008; Park et al., 2012; Bertrand and Marsh, 2015; Bolhuis et al., 2016; Datnow et al., 2020). Case studies such as Selwyn (2020), which combined extensive ethnographic research with software-based methods like data log analysis or data-network mapping (see Selwyn, 2020), were the exception in our analyses.
Regarding the studies that reported a mixed-method approach (Bertrand and Marsh, 2015; Jimerson et al., 2015; Ebbeler et al., 2017; Mandinach and Miskell, 2017a; Hubers et al., 2018), it can be noted that they equally addressed issues from leadership-related and teacher-related perspectives. What is striking about these studies, however, is that despite the mixed-method approach, foci are apparent in terms of the methods chosen (e.g., Farley-Ripple and Buttram, 2013; Jimerson and Reames, 2015; Ebbeler et al., 2017; Hubers et al., 2018). For example, Abrams et al. (2016) report that their study is embedded in a larger mixed-method research, but the reporting focus is clearly on the results of qualitative focus group interviews (Abrams et al., 2016). Light et al. (2005) also report the use of both qualitative and quantitative methods, with a clear emphasis on the former. Thus, mainly interviews and observations are used, which are only supplemented or enriched by questionnaires.
With regard to the research methods used, it can be stated, as for the case studies before, that mainly traditional research methods are also used within the framework of the included quantitative and qualitative studies. These are questionnaires (e.g., Mcmillan et al., 2010; Schildkamp and Visscher, 2010b; Ebbeler et al., 2017; Hubers et al., 2018; Prenger and Schildkamp, 2018) on the side of quantitative approaches and interviews (e.g., Wohlstetter et al., 2008; Park and Datnow, 2009; Datnow et al., 2012) on the side of qualitative approaches. In contrast, primary designs that support a practice theory perspective (e.g., for ethnographic studies, see Lockton et al., 2018; Selwyn, 2020) are seldom found.
With regard to the practical implementation of the research methods, we were also able to observe an enormous variation. For example, information on how “observations” were conducted varied: Some studies reported “multiday site visits” (e.g., Kennedy and Datnow, 2011; Park and Datnow, 2017; Selwyn, 2020), while others simply used the term “observation” (Wayman and Jimerson, 2014; Bertrand and Marsh, 2015). However, a detailed explanation of the methodological approach, for example, the degree of structuring and openness of the observation, is mostly lacking.
By going a step further and analyzing the theoretical lenses described and used in the studies, we were able to elaborate our mapping in more detail. On the left side of the spectrum, representing quantitative designs in our mapping, we were able to identify (mostly) psychological factors such as affective and instrumental attitudes, perceived control, social norms, and self- or collective efficacy. On the other side of the spectrum, we found methodological studies that addressed aspects such as sense-making, (distributed) leadership theories, and co-creation theory.
Discussion
In summary, after analyzing previous studies, we can say that there are many primarily exploratory research designs in the area of data practices in the school context. These were particularly evident from the many case studies published during the period we examined. Exploratory research designs are used to gain an initial overview of a phenomenon or field. Thus, it stands to reason that there has been little long-term research on the topic of data practices in schools, particularly from the perspective of social practices and with reference to digital data, which was our primary interest. This is certainly due in part to the fact that most studies focus on examining data-driven decision making (DDDM) in schools or school evaluation, and thus perspectives of evidence-based educational research or school governance (for dddm see, e.g., Mandinach, 2012; Mandinach and Gummer, 2013; Schildkamp et al., 2015, 2019a; Datnow and Hubbard, 2016). The focus of these studies is then logically on both data infrastructures and/or their impact on the organizational culture of schools and the related development of collaboration, communication, and organizational trust. From our perspective, however, an interesting correlation emerges from our overarching look at the publication landscape: most studies on organizational and leadership levels are quantitative in nature. While this can be inferred primarily from the content focus, it invites further methodological and methodological considerations: In particular, with regard to data-driven decision making (DDDM), qualitative studies could go beyond existing research perspectives. Here, it would be possible to examine practices of such decision-making processes in schools and offer deeper insights into these processes than simply examining the frequency of data use and the nature of the data. These insights could in turn inform school development and management issues. In analyzing the research methodological approach of the studies, we generally find that the studies to date have followed more traditional research methods. For example, we find only few ethnographic studies that would provide insights into how “data collection” is conducted in schools and, in particular, in the context of pedagogical practices.
In addition, we found other blind spots when looking at the studies: Studies that explicitly address digital data and related practices seem to be sporadic until 2019, despite the fact that the process of increasing datafication has implications for schools as well. In this regard, digital data from educational applications and platforms, for example, are becoming increasingly important for pedagogical practice and decision-making, also beyond debates about AI in schools. Dashboards in learning management systems make it possible to quickly assess student learning “at a glance” and provide appropriate feedback.
The basic idea here is to show users what is most important ‘at a glance,’ so that data no longer has to be laboriously compiled or interpreted to derive decisions for action. The argument here is usually time savings, but also the problem that non-data experts should also be able to use the tools ‘correctly’. The problem behind this is: the ‘clearer’ the picture (keyword traffic light system), the faster and easier it is to grasp., the more is already selected and informed, the more is not mapped (Hartong, 2019, p. 14).
Considering the impact of digital data on instructional design and other pedagogical practices thus seems not only worthwhile, but necessary in the future, as new inequalities come into play with such decision-making processes. In summary it must be stated that the research focus of the articles we found and analyzed so far is primarily on data use and its effects in everyday school life, in order to be able to make evidence-based decisions based on it. A perspective on data practices as social practices and “doing data” is hard to find. However, the results presented here have limitations due to the chosen methodology and the approach to the literature search: for example, the defined inclusion criteria led to a restriction of the papers included in the analyses, e.g., those that are not published in English or German or those that we could not find with the search terms we used. Also, the time period may have been too early for the perspective we were interested in, in that we looked at articles published before 2020. In this context, the analyses also rely on a relatively small data corpus. Although the subsequent snowball search attempted to fill gaps in the data corpus, the results cannot claim to be exhaustive. Nonetheless, the overview provided here offers implications for future research that may help to map previous blind spots in research on data practices in schools and address them through new studies.
Author contributions
UK, AH, and MS-R examined the articles to address research questions and wrote the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.
Funding
This work was supported by the Federal Ministry of Education and Research Germany under grant 01JD1903A/B/C.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Abrams, L., Varier, D., and Jackson, L. (2016). Unpacking instructional alignment: the influence of teachers’ use of assessment data on instruction. Perspect. Educ. 34, 15–28. doi: 10.18820/2519593X/pie.v34i4.2
Allert, H., and Richter, C. (2017). Kultur der Digitalität statt digitaler Bildungsrevolution. Pädagogische Rundschau 71, 19–32.
Anagnostopoulos, D., Rutledge, S. A., and Jacobsen, R. (2013). The infrastructure of accountability: Data use and the transformation of American education. eds. D. Anagnostopoulos, S. A. Rutledge, and R. Jacobsen (Cambridge: Harvard Education Press).
Bertrand, M., and Marsh, J. A. (2015). Teachers’ Sensemaking of data and implications for equity. Am. Educ. Res. J. 52, 861–893. doi: 10.3102/0002831215599251
Bolhuis, E. D., Schildkamp, K., and Voogt, J. M. (2016). Improving teacher education in Netherlands: data team as learning team? Eur. J. Teach. Educ. 39, 320–339. doi: 10.1080/02619768.2016.1171313
Bowers, A. J. (2009). Reconsidering grades as data for decision making: more than just academic knowledge. J. Educ. Adm. 47, 609–629. doi: 10.1108/09578230910981080
Bowers, A. J. (2016). Quantitative research methods training in education leadership and administration preparation programs as disciplined inquiry for building school improvement capacity. J. Res. Lead. Educ. 12, 72–96. doi: 10.1177/1942775116659462
Breiter, A., and Hepp, A. (2018). “The complexity of datafication: putting digital traces in context” in Communicative figurations. eds. A. Hepp, A. Breiter, and U. Hasebrink (Cham: Springer International Publishing), 387–405.
Brown, C., Schildkamp, K., and Hubers, M. D. (2017). Combining the best of two worlds: a conceptual proposal for evidence-informed school improvement. Educ. Res. 59, 154–172. doi: 10.1080/00131881.2017.1304327
Coburn, C. E., and Talbert, J. E. (2006). Conceptions of evidence use in school districts: mapping the terrain. Am. J. Educ. 112, 469–495. doi: 10.1086/505056
Coburn, C. E., and Turner, E. O. (2011). Research on data use: a framework and analysis. Meas. Interdis, Res. Persp. 9, 173–206. doi: 10.1080/15366367.2011.626729
Dander, V., and Aßmann, S. (2015). “Medienpädagogik Und (Big) Data: Konsequenzen Für Die Erziehungswissenschaftliche Medienforschung Und -Praxis” in Big Data Und Medienbildung, Schriftenreihe Zur Digitalen Gesellschaft NRW. ed. H. Gapski (München: kopaed verlagsGmbH), 33–50.
Datnow, A., and Hubbard, L. (2016). Teacher capacity for and beliefs about data-driven decision making: a literature review of international research. J. Educ. Chang. 17, 7–28. doi: 10.1007/s10833-015-9264-2
Datnow, A., Lockton, M., and Weddle, H. (2020). Redefining or reinforcing accountability? An examination of meeting routines in schools. J. Educ. Chang. 21, 109–134. doi: 10.1007/s10833-019-09349-z
Datnow, A., and Park, V. (2018). Opening or closing doors for students? Equity and data use in schools. J. Educ. Chang. 19, 131–152. doi: 10.1007/s10833-018-9323-6
Datnow, A., Park, V., and Kennedy-Lewis, B. (2012). High school Teachers' use of data to inform instruction. JESPAR 17, 247–265. doi: 10.1080/10824669.2012.718944
Datnow, A., Park, V., and Kennedy-Lewis, B. (2013). Affordances and constraints in the context of teacher collaboration for the purpose of data use. J. Educ. Adm. 51, 341–362. doi: 10.1108/09578231311311500
Decuypere, M. (2021). The topologies of data practices: a methodological introduction. J. New Approach. Educ. Res. 10, 67–84. doi: 10.7821/naer.2021.1.650
der Kleij, V., Fabienne, M., Vermeulen, J. A., Schildkamp, K., and Eggen, T. J. H. M. (2015). Integrating data-based decision making, assessment for learning and diagnostic testing in formative assessment. Asses. Educ. 22, 324–343. doi: 10.1080/0969594X.2014.999024
Ebbeler, J., Poortman, C. L., Schildkamp, K., and Pieters, J. M. (2017). The effects of a data use intervention on educators’ satisfaction and data literacy. Educ. Assess. Eval. Account. 29, 83–105. doi: 10.1007/s11092-016-9251-z
Ercan, H., Hartmann, U., Richter, D., Kuschel, J., and Gräsel, C. (2021). Effekte von integrativer Führung auf die Datennutzung von Lehrkräften. Die deutsche Schule 113, 85–100. doi: 10.25656/01:22077
Espeland, W. N., and Stevens, M. L. (2008). A sociology of quantification. Eur. J. Sociol. 49, 401–436. doi: 10.1017/S0003975609000150
Espin, C. A., Wayman, M. M., Deno, S. L., McMaster, K. L., and de Rooij, M. (2017). Data-based decision-making: developing a method for capturing teachers’ understanding of CBM graphs. Learn. Disabil. Res. Pract. 32, 8–21. doi: 10.1111/ldrp.12123
Farley-Ripple, E. N., and Buttram, J. L. (2013). Harnessing the power of teacher networks. Phi Delta Kappan 95, 12–15. doi: 10.1177/003172171309500304
Förschler, A., Hartong, S., Kramer, A., Meister-Scheytt, C., and Junne, J. (2021). “Zur (ambivalenten) Wirkmächtigkeit Datengetriebener Lernplattformen: Eine Analyse Des ‹Antolin›-Leseförderungsprogramms”. MedienPädagogik. Zeitschrift für Theorie Und Praxis Der Medienbildung 44, 52–72. doi: 10.21240/mpaed/44/2021.10.28.X
Gulson, K. N., Lewis, S., Lingard, B., Lubienski, C., Takayama, K., and Taylor Webb, P. (2017). Policy mobilities and methodology: a proposition for inventive methods in education policy studies. Crit. Stud. Educ. 58, 224–241. doi: 10.1080/17508487.2017.1288150
Gulson, K. N., and Sellar, S. (2019). Emerging data infrastructures and the new topologies of education policy. Environ. Plan. D 37, 350–366. doi: 10.1177/0263775818813144
Hartong, S. (2019). Bildung 4.0? Kritische Überlegungen zur Digitalisierung von Bildung als erziehungswissenschaftliches Forschungsfeld (Bildung 4.0? Critical Reflections on the Digitization of Education as a Field of Research in Educational Science). Zeitschrift für Pädagogik 65, 424–444. doi: 10.25656/01:23950
Honig, M. I., and Coburn, C. (2008). Evidence-based decision making in School District central offices: toward a policy and research agenda. Educ. Policy 22, 578–608. doi: 10.1177/0895904807307067
Honig, M. I., and Venkateswaran, N. (2012). School–central office relationships in evidence use: understanding evidence use as a systems problem. Am. J. Educ. 118, 199–222. doi: 10.1086/663282
Houben, Daniel, and Prietl, Bianca. (2018). Datengesellschaft: Einsichten in die Datafizierung des Sozialen. eds. D. Houben and B. Prietl (Bielefeld: transcript-Verlag).
Hubers, M. D., Moolenaar, N. M., Schildkamp, K., Daly, A. J., Handelzalts, A., and Pieters, J. M. (2018). Share and succeed: the development of knowledge sharing and brokerage in data teams’ network structures. Res. Pap. Educ. 33, 216–238. doi: 10.1080/02671522.2017.1286682
Jarke, J., and Breiter, A. (2019). Editorial: the datafication of education. Learn. Media Technol. 44, 1–6. doi: 10.1080/17439884.2019.1573833
Jimerson, J. B. (2016). How are we approaching data-informed practice? Development of the survey of data use and professional learning. Educ. Assess. Eval. Account. 28, 61–87. doi: 10.1007/s11092-015-9222-9
Jimerson, J. B., Cho, V., Scroggins, K. A., Balial, R., and Robinson, R. R. (2018). How and why teachers engage students with data. Educ. Stud. 45, 667–691. doi: 10.1080/03055698.2018.1509781
Jimerson, J. B., Choate, M. R., and Dietz, L. K. (2015). Supporting data-informed practice among early career teachers: the role of mentors. Leadersh. Policy Sch. 14, 204–232. doi: 10.1080/15700763.2014.997937
Jimerson, J. B., and Reames, E. (2015). Student-involved data use: establishing the evidence base. J. Educ. Chang. 16, 281–304. doi: 10.1007/s10833-015-9246-4
Kalthoff, Herbert, Cress, Torsten, and Röhl, Tobias. (eds.) (2016). Materialität: Herausforderungen Für Die Sozial- und Kulturwissenschaften. Paderborn: Wilhelm Fink Verlag, doi: 10.30965/9783846757048
Kennedy, B. L., and Datnow, A. (2011). Student involvement and data-driven decision making. Youth Soc. 43, 1246–1271. doi: 10.1177/0044118X10388219
Kitchin, R. (2016). Thinking critically about and researching algorithms. Inf. Commun. Soc. 20, 14–29. doi: 10.1080/1369118X.2016.1154087
Krein, U., and Schiefner-Rohs., M. (2021). Data in schools: (changing) practices and blind spots at a glance. Front. Educ. 6, 1–13. doi: 10.3389/feduc.2021.672666
Lasater, K., Albiladi., W. S., Davis., W. S., and Bengtson., E. (2020). The data culture continuum: an examination of school data cultures. Educ. Adm. Q. 56, 533–569. doi: 10.1177/0013161X19873034
Levin, J. A., and Datnow, A. (2012). The principal role in data-driven decision making: using case-study data to develop multi-mediator models of educational reform. Sch. Eff. Sch. Improv. 23, 179–201. doi: 10.1080/09243453.2011.599394
Light, Daniel, Honey, Margaret, Heinze, Juliet, and Brunner, Cornelia. (2005). Linking data and learning–the grow network study: Summary report. New York: EDC's Center for Children and Technology.
Lockton, M., Weddle, H., and Datnow, A. (2018). When data don’t drive: teacher agency in data use efforts in low-performing schools. Sch. Eff. Sch. Improv. 31, 243–265. doi: 10.1080/09243453.2019.1647442
Makkonen, Reino, and Crane, Eric. (2019). What California educators are saying about data use and continuous improvement. San Francisco: WestEd.
Mandinach, E. B. (2012). A perfect time for data use: using data-driven decision making to inform practice. Educ. Psychol. 47, 71–85. doi: 10.1080/00461520.2012.667064
Mandinach, E. B., and Gummer, E. S. (2013). A systemic view of implementing data literacy in educator preparation. Educ. Res. 42, 30–37. doi: 10.3102/0013189X12459803
Mandinach, E. B., and Miskell, R. C. (2017a). Blended learning and data use in three technology-infused charter schools. LEARNing Landscapes 11, 183–198. doi: 10.36510/learnland.v11i1.931
Mandinach, Ellen B., and Miskell, Ryan C.. (2017b). Focus groups to support the development and utility of EdWise for parental stakeholders: Findings from the parental focus groups. San Francisco: WestEd.
Mau, Steffen. (eds.) (2017). Das metrische Wir: Über die Quantifizierung des Sozialen. Berlin: Suhrkamp.
Mcmillan, J., and H., Jessye Cohen, and Lisa Abrams., (2010). Understanding secondary Teachers' formative assessment practices and their relationship to student motivation. Virginia Commonwealth University: ERIC Clearinghouse.
Newman, M., and Gough, D. (2020). “Systematic reviews in educational research: methodology, perspectives and application,” in Systematic reviews in educational research: Methodology, perspectives and application. eds. O. Zawacki-Richter, M. Kerres, and S. Bedenlier (Wiesbaden: Springer VS), 3–22. doi: 10.1007/978-3-658-27602-7_1
Park, V., Daly, A. J., and Guerra, A. W. (2012). Strategic framing: how leaders craft the meaning of data use for equity and learning. Educ. Policy 27, 645–675. doi: 10.1177/0895904811429295
Park, V., and Datnow, A. (2009). Co-constructing distributed leadership: district and school connectionsin data-driven decision-making. School Leadersh. Manag. 29, 477–494. doi: 10.1080/13632430903162541
Park, V., and Datnow, A. (2017). Ability grouping and differentiated instruction in an era of data-driven decision making. Am. J. Educ. 123, 281–306. doi: 10.1086/689930
Park, V., John, E. S., Datnow, A., and Choi, B. (2017). The balancing act. J. Educ. Adm. 55, 390–406. doi: 10.1108/JEA-09-2016-0098
Prenger, R., and Schildkamp, K. (2018). Data-based decision making for teacher and student learning: a psychological perspective on the role of the teacher. Educ. Psychol. 38, 734–752. doi: 10.1080/01443410.2018.1426834
Reckwitz, A. (2003). Grundelemente einer Theorie sozialer Praktiken / basic elements of a theory of social practices. Z. Soziol. 32, 282–301. doi: 10.1515/zfsoz-2003-0401
Röhl, Tobias. (eds.) (2013). Dinge des Wissens: Schulunterricht als sozio-materielle Praxis. Berlin: De Gruyter, doi: 10.1515/9783110507263
Schatzki, Theodore R. (eds.) (1996). Social Practices. Cambridge: University Press, doi: 10.1017/CBO9780511527470
Schildkamp, K. (2019). Data-based decision-making for school improvement: research insights and gaps. Educ. Res. 61, 257–273. doi: 10.1080/00131881.2019.1625716
Schildkamp, K., and Kuiper, W. (2010). Data-informed curriculum reform: which data, what purposes, and promoting and hindering factors. Teach. Teach. Educ. 26, 482–496. doi: 10.1016/j.tate.2009.06.007
Schildkamp, K., Poortman, C. L., Ebbeler, J., and Pieters, J. M. (2019a). How school leaders can build effective data teams: five building blocks for a new wave of data-informed decision making. J. Educ. Chang. 20, 283–325. doi: 10.1007/s10833-019-09345-3
Schildkamp, K., Poortman, C. L., and Handelzalts, A. (2015). Data teams for school improvement. Sch. Eff. Sch. Improv. 27, 228–254. doi: 10.1080/09243453.2015.1056192
Schildkamp, K., Poortman, C., Luyten, H., and Ebbeler, J. (2017). Factors promoting and hindering data-based decision making in schools. Sch. Eff. Sch. Improv. 28, 242–258. doi: 10.1080/09243453.2016.1256901
Schildkamp, K., Rekers-Mombarg, L. T. M., and Harms, T. J. (2012a). Student group differences in examination results and utilization for policy and school development. Sch. Eff. Sch. Improv. 23, 229–255. doi: 10.1080/09243453.2011.652123
Schildkamp, K., Smit, M., and Blossing, U. (2019b). Professional development in the use of data: from data to knowledge in data teams. Scand. J. Educ. Res. 63, 393–411. doi: 10.1080/00313831.2017.1376350
Schildkamp, K., and Teddlie, C. (2007). School performance feedback systems in the United States and in Netherlands: a comparison. Educ. Res. Eval. 14, 255–282. doi: 10.1080/13803610802048874
Schildkamp, K., Vanhoof, J., van Petegem, P., and Visscher, A. (2012b). The use of school self-evaluation results in Netherlands and Flanders. Br. Educ. Res. J. 38, 125–152. doi: 10.1080/01411926.2010.528556
Schildkamp, K., and Visscher, A. (2009). Factors influencing the utilisation of a school self-evaluation instrument. Stud. Educ. Eval. 35, 150–159. doi: 10.1016/j.stueduc.2009.12.001
Schildkamp, K., and Visscher, A. (2010a). The use of performance feedback in school improvement in Louisiana. Teach. Teach. Educ. 26, 1389–1403. doi: 10.1016/j.tate.2010.04.004
Schildkamp, K., and Visscher, A. (2010b). The utilisation of a school self-evaluation instrument. Educ. Stud. 36, 371–389. doi: 10.1080/03055690903424741
Schildkamp, K., Visscher, A., and Luyten, H. (2009). The effects of the use of a school self-evaluation instrument. Sch. Eff. Sch. Improv. 20, 69–88. doi: 10.1080/09243450802605506
Selwyn, N. (2015). Data entry: towards the critical study of digital data and education. Learn. Media Technol. 40, 64–82. doi: 10.1080/17439884.2014.921628
Selwyn, N. (2020). ‘Just playing around with excel and pivot tables’–the realities of data-driven schooling. Res. Pap. Educ. 37, 95–114. doi: 10.1080/02671522.2020.1812107
Selwyn, N., Henderson, M., and Chao, S.-H. (2015). Exploring the role of digital data in contemporary schools and schooling-‘200,000 lines in an excel spreadsheet’. Br. Educ. Res. J. 41, 767–781. doi: 10.1002/berj.3186
Selwyn, N., and Pangrazio, L. (2018). Doing data differently? Developing personal data tactics and strategies amongst young mobile media users. Big Data Soc. 5, 205395171876502–205395171876512. doi: 10.1177/2053951718765021
Selwyn, N., Pangrazio, L., and Cumbo, B. (2021). Attending to data: exploring the use of attendance data within the datafied school. Res. Educ. 109, 72–89. doi: 10.1177/0034523720984200
Smith, G. J. D. (2018). Data doxa: the affective consequences of data practices. Big Data Soc. 5, 205395171775155–205395171775115. doi: 10.1177/2053951717751551
Spillane, J. P. (2012). Data in practice: conceptualizing the data-based decision-making phenomena. Am. J. Educ. 118, 113–141. doi: 10.1086/663283
Sun, J., Przybylski, R., and Johnson, B. J. (2016). A review of research on teachers’ use of student data: from the perspective of school leadership. Educ. Assess. Eval. Account. 28, 5–33. doi: 10.1007/s11092-016-9238-9
Taveras, Barbara, Douwes, Casissa, and Johnson, Karen. (2010). New visions for public schools: Using data to engage families. Cambridge: Harvard Family Research Project.
van de Oudeweetering, K., and Decuypere, M. (2021). In between hyperboles: forms and formations in open education. Learn. Media Technol. 46, 60–77. doi: 10.1080/17439884.2020.1809451
Vanlommel, K., and Schildkamp, K. (2019). How do teachers make sense of data in the context of high-stakes decision making? Am. Educ. Res. J. 56, 792–821. doi: 10.3102/0002831218803891
Wayman, J. C., and Jimerson, J. B. (2014). Teacher needs for data-related professional learning. Stud. Educ. Eval. 42, 25–34. doi: 10.1016/j.stueduc.2013.11.001
Wayman, J. C., Jimerson, J. B., and Cho, V. (2012). Organizational considerations in establishing the data-Informed District. Sch. Eff. Sch. Improv. 23, 159–178. doi: 10.1080/09243453.2011.652124
Wohlin, C. (2014). “Guidelines for snowballing in systematic literature studies and a replication in software engineering,” in Proceedings of the 18th international conference on evaluation and assessment in software engineering–EASE ‘14 (New York: ACM Press) doi: 10.1145/2601248.2601268
Keywords: data, data practices, review, school, social science
Citation: Krein U, Hartenstein A and Schiefner-Rohs M (2022) Mapping research approaches to data practices in schools. Front. Educ. 7:1023096. doi: 10.3389/feduc.2022.1023096
Edited by:
Koorosh Gharehbaghi, RMIT University, AustraliaReviewed by:
Marsha E. Modeste, The Pennsylvania State University (PSU), United StatesAmin Hosseinian-Far, University of Northampton, United Kingdom
Ken Farnes, RMIT University, Australia
Copyright © 2022 Krein, Hartenstein and Schiefner-Rohs. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Ulrike Krein, ulrike.krein@sowi.uni-kl.de