Interdisciplinary Doctoral Training in Technology-Enhanced Learning in Europe

Technology enhanced learning (TEL) research connects Learning Sciences, Educational Psychology, and Computer Science, in order to investigate interventions based on digital technologies in education and training settings. In this paper, we argue that doctoral training activity for TEL needs to be situated at the intersection of disciplines in order to facilitate innovation. For this, we first review the state of disciplinarity in TEL, reviewing existing meta-studies of the field. Then, we survey 35 doctoral education programs in Europe in which doctoral students working on TEL topics are enrolled. Findings indicate that most doctoral schools are associated with a single discipline and offer methodological rather than content-specific modules. TEL-specific content is provided only in exceptional cases, creating a potentially isolating gap between master-level education and scientific conferences. On this background, we argue that cross-institutional doctoral training is important to progress TEL as a field. In this article, we study and share the approach of an international doctoral summer school organized by the European society EA-TEL over the past 15 years. The summer school provides foundational methodological knowledge from multiple disciplines, content-specific topical knowledge in TEL, access to cutting edge scientific discourse, and discussion of horizontal issues to doctoral students. We further provide an analysis of shifting program topics over time. Our analysis of both, institutional as well as cross-institutional doctoral training in TEL, constitutes this paper’s core contribution in that it highlights that further integration of perspectives and knowledge is to be done in TEL; together with codification and explication of knowledge in the intersection of disciplines.

Introduction

Research fields are, in many ways, set up as communities of shared knowledge and practice (Lave and Wenger, 1995; Latour, 2005), typically geographically distributed. Communities differentiate themselves from each other with regards to what the agreed objects of interest are, and what are to be considered valid ways of contributing and gaining seniority (Lave and Wenger, 1995; Kuhn, 2012). This includes specific methodological commitments in extension of a generally shared agreement across disciplines that the generation of new knowledge is the goal. Moreover, this also involves an often unspoken agreement as to which publishing venues are considered acceptable and reputable. Doctoral training is often considered an academic rite of passage (Amran and Ibrahim, 2012).

Research fields tend to cascade into Higher Education over time, for instance in the form of doctoral schools, as a way to commodify recruitment and training of future community members. Doctoral education is thereby commonly implemented in non-interdisciplinary academic structures (Lindvig, 2018), while at the same time aiming to establish a transdisciplinary view of science (“mode 2 science”), driven by grand challenges (O’Rourke et al., 2016) that do not regard disciplinary boundaries (Carr et al., 2018).

In principle, one could discuss that “doctoral-level education” (as in “doctoral training program” or “Ph.D. studies”) is an oxymoron, as any such expression pretends that the key principles of education could be directly applied to research. Any common definition of ‘education’ includes the idea of giving and receiving systematic instruction to motivate the re-construction or re-development of existing knowledge, skills, abilities, and other characteristics by the recipient of education, the learner, of course adapted to given context. Even more thought-provoking, ideas of academic knowledge exchange suggest that skills should be transferred from a knowledgeable scholar (and their academic outputs such as textbooks, journal articles, or online course materials). “Research” on the other hand requires systematic investigation, with the aim to discover or develop a novel insight, previously unknown. Delineating it from bachelor (level 6) and master (level 7), the International Standard Classification of Education speaks in this context for its definition of level 8 of requiring submission of “written work of publishable quality that is the product of original research and represents a significant contribution to knowledge in the respective field of study” (ISCED, 2011, p. 60, § 264).

The review of the state of the art, however, has become and will become increasingly more complex, as the amount of codified knowledge (publications, research data) grows continuously year after year. In parallel, methods evolve to take up new possibilities to analyze data, and to do so in a more complex manner. For example, public betas (“facebook as a testbed”), open test collections, online crowdsourcing, and participatory approaches such as citizen science promise to lower barriers to research (regarding access, replication, and reuse, see Cleverdon, 1960; Kittur et al., 2008; Shneiderman, 2008; Herodotou et al., 2014). New requirements emerge regarding ethics, research and research data documentation, and accessibility. From this position, one could argue that as both methodology, practice, and existing knowledge exhibit increased complexity when operated on, there is a need for additional training and guidance beyond the prerequisite bachelor and master levels.

Nevertheless, doctoral training is widely accepted to be a key activity of research communities. Technology Enhanced Learning (TEL) is no exception to this. This article therefore sets out to deliver both an analysis of the current governing structure of doctoral education in TEL, particularly in Europe, and a proposal for a common core of doctoral-level training in TEL. We break this further down into the following research questions:

• RQ1: What is the current practice of institutional doctoral training in TEL in Europe?

• RQ2: How could cross-institutional doctoral education be organized, and which topics are relevant?

We first investigate the state of affairs with regards to the disciplinarity in the field of TEL as background to our present discussion (see section “Technology Enhanced Learning as an Interdiscipline”). Then we describe our methodological approach to answering the above two research questions (see section “Methodology”). In section “Ph.D. programs in a Single Department or Doctoral School” we describe the heterogeneity of current doctoral training in TEL at European universities based on a survey, and present an overview on the past 15 years of the historical development of cross-institutional and interdisciplinary doctoral school in the framework of what is now the European Association of Technology Enhanced Learning (EA-TEL¹). Finally, we conclude, also with an outline a vision for further development of the framework and connected measures of success (see section “Dedicated Doctoral Training in an International Society”).

Technology Enhanced Learning as an Interdiscipline

Technology enhanced learning is an interdisciplinary field that connects Computer Science with the Learning Sciences, Psychology, and other Social Sciences, Humanities, or Engineering Sciences (Meyer, 2011; Tchounikine, 2011; Meyer et al., 2013; Kalz and Specht, 2014). Wild (2016) defines TEL as being directed at “human development of competence […] with tools that afford isolated or collaborative endeavors in formal and informal situations”, deliberately defining TEL as inclusive for both educational as well as professional contexts.

While TEL is a standing term in European research, sometimes its related expressions are preferred internationally, such as Educational Technology, Digital Education, and Learning Engineering (see Figure 1 for a depiction of regional preferences in terminology). The expression of Learning Engineering recognizes the need for technical competence as an essential requirement for learning and development initiatives in fields that methodologically depend on data science, Computer Science, and Learning Sciences.

FIGURE 1

Figure 1. Google search trends of competing terms by region (07/2010–07/2020). Data source: Google Trends (https://www.google. com/trends). Color intensity represents percentage of searches of most searched of the three terms (the higher the percentage, the more intense the color). Percentage means percentage of all searches of the three terms.

All terms and definitions recognize the need of epistemic fluency to facilitate interdisciplinary dynamics, in which participating professionals have “the capacity to understand, switch between, and combine different kinds of knowledge and different ways of knowing” (Markauskaite and Goodyear, 2017).

Examples of Interdisciplinary Dynamics in TEL Research

To discuss aspects of interdisciplinarity for Technology enhanced learning (TEL), we first need to operationalize the different terms commonly used to describe collaboration between scientific actors. For this purpose, we build on the work of Wagner et al. (2011) who provided the definitions listed in Table 1.

TABLE 1

Table 1. Definitions based on Wagner et al. (2011).

Furthermore Kalz and Specht (2014) differentiate four distinct approaches to interdisciplinary research, based on the work of Aboelela et al. (2007):

• Interaction-oriented: How do members of a scientific community cooperate?

• Communication-oriented: How do members talk to each other?

• Conceptual: How are concepts, ideas and models integrated in the inquiry-phase for problem-solving?

• Outcome-oriented: What are the products of the cooperation?

In this article, we understand and discuss TEL as interdisciplinary, since actors jointly address the question how technologies can be used to make learning more effective, efficient, enjoyable, or accessible. In addition, we follow in this study an outcome-oriented approach combined with a conceptual approach.

Technology Enhanced Learning (TEL) research is often connected to practical problems or “grand challenges” of education, a theory, or technological affordances. For example, it is well known that the most effective way for humans to acquire domain knowledge is by 1:1 tutoring. At the same time, however, it is simply not possible to offer a private tutor to each student, posing a grand challenge. Such practical problems very often make integration of knowledge from different disciplines necessary. In this sense, TEL is an interesting case study for an analysis of interdisciplinarity, since the work profits from mono-disciplinary research of the contributing domains while at the same time problems of TEL can only be addressed by joint work. The below examples illustrate TEL interdisciplinarity, and the feedback to the disciplines from a conceptual and outcome-oriented perspective.

As a solution to the tutoring problem, Personalized Adaptive Learning Systems have been conceived in the Computer Science field, using models of learning and cognitive science as input to their design. As a popular example, the “cognitive tutor” (e.g., Ritter et al., 2007) has been built around models of cognitive psychology derived from ACT-R, a general purpose cognitive architecture that explains the working of different cognitive functions like perception, memory and learning. Interdisciplinarity goes even a step further, namely when the results of the cognitive tutor’s evaluation in practice is fed back to the contributing disciplines: In math education, the construction and ordering of problems and the creation of curricula has been influenced (e.g., Ritter et al., 2007). In cognitive psychology, data from large scale evaluation can now be used to validate models derived from the cognitive psychology that are commonly only studied in the laboratory, such as how concepts are formed in self-directed learning (Seitlinger et al., 2020). For intelligent systems, some general implications have been derived in terms of what models are involved (e.g., knowledge base, learner model, adaptation model), how they can be formalized and applied, e.g., by providing adaptive prompts for reflection (Fessl et al., 2017).

Another example for interdisciplinary dynamics in TEL can be found in research on organizational workplace collaboration and learning. By analyzing technologies used for collaboration and the resulting collaborative artifacts (e.g., shared notes, wiki pages, ontologies etc.), the knowledge maturation model was established in the Information Systems domain to describe goal directed collective learning processes and how knowledge materialized and matured in a systematic manner in organizational settings, e.g., how initial ideas are transformed into organizational improvements or new products (Maier and Schmidt, 2015). Extending the model from the perspective of the Learning Sciences, the knowledge appropriation model explained how healthcare professionals and construction workers were learning in such settings when they co-created new working practices (e.g., on treating diabetes or on applying sustainable construction techniques) by scaffolding and guiding learning at the workplace (Ley et al., 2019). Finally, the model found application in learning analytics, where it was used to analyze traces in an online collaborative learning design environment (Rodríguez-Triana et al., 2019). This allowed insights into collaborative learning and design processes that otherwise would have been difficult to observe, namely that the more teachers build on others’ work, the higher was the likelihood they used the final outcome in the classroom.

Last, but not least, wearable enhanced learning (Buchem et al., 2019) combines technical disciplines, most notably represented through the topics of wearable computing, augmented and virtual reality, artificial intelligence, and machine learning, with social sciences, arts, and humanities, most notably represented through education, design, and social impact studies. It is only in this combination, that research and innovation in learning with wearable technology becomes possible. For example, the sensor-based Augmented Reality system for experience capture and re-enactment documented in Limbu et al. (2019) combines innovative wearable hardware and software technology based on the four-component instructional model 4C/ID that proposes to design for learning by connecting background knowledge with procedural information, and learning and practice tasks (Van Merriënboer, 2019). Another example of interdisciplinary work in wearable enhanced learning can be found in Hall et al. (2019), which combines insights about post-stroke rehabilitation from the health sciences with a feedback model from the neuro and learning sciences in its computer science and audio engineering implementation of a real-time auditory biofeedback system for (re-)learning arm trajectories.

Meta-Analysis of the Dynamics of Interdisciplinarity in the TEL Research Community

Several outcome-oriented studies have been conducted in the TEL field that have focused on publications, sometimes combined with an interaction-oriented approach. Dynamics in the research field were investigated more broadly in the past, by applying scientometric analyses or the analysis of research collaboration and funding. While these studies are now dated and were conducted in a comparatively short time-frame (publication dates 2012-2014), an artifact of funding policy, they provide evidence of how the field emerged and developed over time.

Kalz and Specht (2014) applied a publication analysis using 3,746 TEL publications indexed in the Web-of-Science. By comparing within-domain with outside-domain citations as the measure of diversity (weighted by disparity/variety of disciplines participating), the authors conclude that the field operates on a high level of interdisciplinarity.

Meyer et al. (2013) studied interdisciplinarity and research practice in TEL using a survey (N = 123), complemented with a social network analysis over publicly available information on research collaboration of the participants (who were not anonymized). The authors found diverse disciplinary backgrounds among the respondents, including social sciences, engineering, multidisciplinary backgrounds, and backgrounds in other disciplines such as life and natural sciences. A cluster analysis identified key groups among the respondents, differentiating along two axis, namely degree of TEL participation and disciplinary orientation. The motor of the community, i.e., those groups with high participation in TEL and an interdisciplinary view, is identified in three groups: established computer scientists (5%), TEL interdisciplinarians (21%), and progressive social scientists (10%). The social network analysis added to that picture that the TEL interdisciplinarians show the highest betweenness centrality value, indicating that “many others are dependent on this group in order to reach indirect contacts” (Meyer et al., 2013).

Pham et al. (2012) analyzed five major TEL conferences with the help of social network analysis, ICALT, AIED, EC-TEL, ITS, and ICWL. AIED and ITS exhibit mature, so-called ‘focused’ author communities with stable, in parts hierarchical structure and few isolates. Both conferences bridge disciplines and bring together artificial intelligence research with research in education. The publications at ICWL and ICALT are less connected than those of ITS, AIED, and ECTEL. Both ICWL and ICALT are inclusive and open to a wide range of perspectives, at the same time reflecting fragmentation of their global constituencies. This is supported by analyzing the development of the maximum betweenness values. This indicates the existence of more common core references in the scientific communities of ITS, AIED, and ECTEL. The diameters of ECTEL and AIED have begun to shrink very early, indicating that the body of literature of these communities is relatively stable and the themes of the communities are settled, reflecting the common ground that exists by now. Overall, for most TEL conferences (not ICWL) at the time, more than 35% authors continued to publish at the same venue.

Derntl and Klamma (2012) analyzed European project funding in TEL, and found in particular European funding for larger research networks (Integrated Projects, and Networks of Excellence in the then current funding program) served to shape the research agenda of the field, and to create strong collaborative ties between research institutions, a characteristic that reflects on the doctoral training offered in these projects.

This snapshot of the community provides evidence for the field’s emergence, maturation, and its interdisciplinarity at the time. It is compelling, especially in times of a Black Swan event, the COVID-19 pandemic, where TEL is more important than ever before, that up-to-date systematic meta-analyses of the TEL research communities’ interactions and collaboration are missing, and therefore, sadly, there are no up-to-date expert and expertise directories. The past analyses, however, establish enough evidence to claim the field as interdisciplinary, even if we may not fully know the current state of affairs with regards to knowledge and community integration, and research communication.

Methodology

In order to answer the first guiding research question (RQ1) about current practices of institutional doctoral training in TEL in Europe, we sampled TEL-oriented programs offered at 35 Higher Education Institutions from eleven countries in Europe. Sampling was expert-driven convenience sampling, collecting data and recommendations from collaborators. Data were extracted from websites and direct communication, looking at content, teaching methodologies, resources, and the administrative context of the programs each. The sample is non representative, but spread out enough to allow for exploration and qualitative insights into the nature of the programs offered. The 35 cases were classified inductively into the three types “Ph.D. programs in a single department or school”, “postgraduate programs (master programs) in a single department or school which offers a TEL specialization”, and “Cross-departmental or multi-disciplinary program.” This analysis investigates to what degree the field of TEL has commodified and institutionalized in form of dedicated doctoral training programs. It is an outcome-oriented analysis, where the unit of analysis is not publications, but educational curricula and administration (association to departments). This analysis is described in section “Ph.D. programs in a Single Department or Doctoral School.”

To answer the second research question (RQ2) about how cross-institutional programs can complement such institutional doctoral training, we study set-up and development of a doctoral summer school, the joint TEL summer school (JTELSS) of the European Association for Technology-Enhanced Learning (EA-TEL). We conduct an in-depth case study of this cross-institutional doctoral training program. The summer school is part of a set of doctoral training activities, complemented by a 2-day Doctoral Consortium and a Ph.D. student best paper award. Both are attached to the annual academic conference of the society, the European Conference on Technology-Enhanced Learning (EC-TEL). This analysis is described in section “Methodology.”

We analyze the programs of 15 years from the perspective of the types of sessions (keynotes, thematic sessions, methodology sessions, soft-skills sessions, informal learning sessions, career-development sessions, see section “EA-TEL Summer School Activity Framework”), and the topics of sessions (see section “Shifting of Program Topics Over Time”). The topics were identified by manual inductive coding (two experts, mediating agreement), with statistical clustering applied over the coded data to identify themes, that were subsequently intelligently labeled. Details of the clustering procedure are described in section “Shifting of Program Topics Over Time” together with the presentation of results. Again, this analysis constitutes an outcome-oriented analysis (distribution session types and topics over time, emergent framework), as well as a conceptual analysis (clustering constituting research themes).

Pertinent Institutional Doctoral Training on TEL in Europe

To answer the first research question about current practice in institutional TEL doctoral training, we summarize below the findings from the survey. The programs that were overall analyzed are listed and categorized are provided as Supplementary Material to this article.

Ph.D. Programs in a Single Department or Doctoral School

Ph.D. Programs in TEL (3 Cases)

An example is the Ph.D. program “Education and ICT (e-learning)” offered by the Open University of Catalonia. This program combines study and research, such that students first get training, and only in a second stage set up and carry out their doctoral research plan. Offered courses in the first phase include both methods (e.g., qualitative and quantitative research methods, or data analytics), and foundations in technology-enhanced learning. In the second phase, an additional personalized study plan is drawn up, while up to five additional blocks of training support progressing the research project (seminars, bridging courses, research/transfer/entrepreneurship courses, workshops).

Monodisciplinary (Ph.D. in Computer Science – 7 Cases; Or in Education – 6 Cases)

In these cases, groups that host TEL doctoral students do research in TEL, but the doctoral programs are not specific to TEL. An example of such a program is at Graz University of Technology, the doctoral school of Computer Science which offers a Ph.D. program in Computer Science and a number of mandatory courses (such as “methods of scientific work”) and elective courses (can be chosen from all university courses at master level, agreed by supervisor/director of studies). Doctoral students carry out their work as part of research groups.

Postgraduate Programs (Master Programs) in a Single Department or School Which Offer a TEL Specialization

Postgraduate Program in TEL (8 Cases)

In these cases, courses are particularly suitable as foundation for a Ph.D. in TEL, and courses connected to relevant research groups / dedicated Ph.D. programs. An example case is the postgraduate course “Educational Technology” offered by the University of Tartu, which provides professional development to people who teach (or plan to teach), and are interested in how to use educational technology in their work. After successful completion, students have the possibility to continue to a Ph.D. program with the same specialization.

Monodisciplinary Postgraduate Program (Computer Science: 7 Cases; Education: 5 Cases)

Cyprus University of Technology, for example, offers a postgraduate program on “Interaction Design” and its graduates can continue their studies for a Ph.D. in various TEL-related areas such as Embodied Play and Learning using Technology, Interaction Design and Creative Collaboration Spaces, Inclusive Design and Social Change using Technology, Design for social change and innovation, and Computer-Assisted Language Learning.

Cross-Departmental or Multidisciplinary Programs

Cross-departmental or multidisciplinary programs (Ph.D.: 2 cases; Postgraduate: 7 cases): An example is at the University of Aveiro which offers a Ph.D. program in “Multimedia in Education,” a joint degree offered by the Communication and Arts and the Department of Education and Psychology.

Summary of the Pertinent Institutional Doctoral Training in Europe

Dedicated Ph.D. and postgraduate programs in TEL [see sections “Ph.D. Programs in TEL (3 Cases)” and “Postgraduate Program in TEL (8 Cases)”] are examples of institutionalization. They are interdisciplinary. Both Ph.D. and postgraduate programs in other fields [see sections “Monodisciplinary (Ph.D. in Computer Science – 7 Cases; or in Education – 6 Cases)” and “Monodisciplinary Postgraduate Program (Computer Science: 7 Cases; Education: 5 Cases)] are examples of non-institutionalized TEL, meaning that they are not institutionalized at all in the respective Higher Education Institution, and training is mono-disciplinary. There are also cross-departmental or multidisciplinary Ph.D. program instances where TEL has been operationalized as cooperation or collaboration between departments and disciplines.

Most doctoral programs investigated focus on methodological courses, rather than on TEL-specific topics or topics specific to another discipline. The programs studied, however, differ widely, meaning they provide so heterogeneous foundational knowledge. From the insights on doctoral training within Higher Education Institutions studied, we have to conclude that the creation of common ground for the field of TEL is not happening from inside these institutions (with limited exceptions). Furthermore, not all Ph.D. programs are available in English, further limiting the sharing of existing resources.

Dedicated Doctoral Training in an International Society

To answer the second research question on how cross-institutional doctoral training could be organized and which topics would be relevant, we have analyzed the program development of a joint European summer school on TEL, organized by EA-TEL. The summer school is typically organized in a rather remote location, so as to underline its retreat character of providing a protective, low-exposure environment for next-generation researchers (compared to a big scientific conference).

The summer school itself is evidence that interdisciplinary Ph.D. training works, managing now for 15 years to bring together next generation researchers with very heterogeneous training, as the application and evaluation processes year after year reveal. Some Ph.D. candidates have no training, while some have a lot of course work. Most Ph.D. candidates attending the summer school are in the early stage of their Ph.D. work (main target group), some are late stage (often co-organizing workshops/part of the organization team). The doctoral consortium is more oriented-toward late stage Ph.D. candidates getting ready for their Ph.D. exam.

In the application process, all Ph.D. candidates are required to submit a summary of their research work. These summaries are peer-reviewed by an international committee of established researchers, following criteria similar to those at academic conferences, such as evaluation of the related work, theoretical framework, methods, and progress. The reviews do not only provide Ph.D. candidates with unbiased feedback, but often urge them to update their own understanding of all components of their work. Most Ph.D. candidates demonstrate a good level of awareness of their selected topic, but many (and not those in the early-stage) are struggling with defining their theoretical frameworks. The evaluation of the methods vary greatly from excellent rates to questioning the overall research design. Some are trying to run a project, lacking research questions, only pursuing development work. Others are misguided to study and reflect upon local TEL efforts in their own institution only, rather than working in a manner conducive to receiving the international recognition (and impact) required for a doctoral degree.

While the doctoral training targets directly Ph.D. candidates, it also indirectly supports Ph.D. advisors. A computer science expert in machine learning could benefit from having a Ph.D. candidate investigating, for example, the design and the impact of educational chatbots liaising with the TEL community. This is a way to stimulate frontier research by supporting Ph.D. advisors in tackling challenges outside their comfort zone.

All three EA-TEL doctoral training instruments share the objective of the society to establish a universal concept of what a Ph.D. in TEL should look like, connecting the community by establishing a review and quality assurance process, reinforcing reflection on how the developed technologies actually serve learning. Review thereby includes both peer review from candidates at other universities as well as from established researchers, using the main conference, EC-TEL, as a recruiting ground. The best student paper award serves as a showcase of what excellence in TEL research looks like.

EA-TEL Summer School Activity Framework

The program of activities offered at the summer school changes annually, reacting to evaluation results of the previous edition, while also implementing new and experimental ideas. Nevertheless, over the years, a stable common framework has emerged, which covers six distinct session types (Figure 2). The program of the summer school is compiled combining sessions selected from submissions to an open call for instructors with ‘standard’ sessions from the framework, added by the organizers. All thematic workshops are proposed via the call for instructors, whereas, typically, all keynotes and informal sessions are added by the organizers. Methodological, soft-skills, and career workshops are mixed-initiative. Some workshops come through via the open call, some are added by the organizers. Ph.D. candidates co-design the program, submitting workshops, often collaboratively with supervisors or peers.

FIGURE 2

Figure 2. Share of total time of different session types against years.

Thematic workshops change year by year, driven by community interest. These workshops serve as indicators not only for the development of the program, but for the TEL field at large. The share of sessions dedicated to these workshops in the program has been stable over the years: in average, 47.8% of the time of the program is spent on thematic workshops (see Figure 2).

Methodological workshops are mostly proposed by the community. They focus on different research methodologies that can be applied in various contexts of TEL research, such as systematic reviews, resign-based research, statistics for TEL, field studies, and many others. They make up for 8.9% of the program in average (time-wise). The participants highly appreciated these workshops and their number grew to 13% in 2019 and 22% in 2020.

Soft-skills workshops remain relatively stable over time, even though they are proposed via the open call. They cover topics such as academic writing, dissemination and communication, or presentation skills, making up for 15.5% of the total time in average.

Established researchers present keynotes covering central themes as well as frontiers topics. In early years, the summer school used “lecture” type sessions submitted to the open call (these sessions were categorized as keynotes for the subsequent analysis). From the early 2010s, instructors were encouraged to focus on interaction rather than lecturing. In 2018, it was decided to accept only interactive workshops via the open call, removing the lecture category from the open submission process, while at the same time increasing the number of keynotes and managing the speakers drafting process centrally through invitation. Since 2018, keynotes are rated higher by the participants than any other session type. Keynotes make up for 15.6% of the total time.

Informal learning sessions have been refined over the years, staying relatively stable in recent years. They include Ice-breaker, Pecha Kucha, Pitch and Poster Session, Fish Bowl, Game Night, and Speed Mentoring. They encourage active participation and allow participants to present their work, bring up their questions and challenges, without any restriction to specific topics. These sessions play a key role in developing strong ties in the community, contributing to the social atmosphere of the event. Informal sessions make up 8.8% of the program.

Career workshops usually target late-stage Ph.D. candidates and focus on opportunities for Ph.D. graduates in both academia and business. They make out 3.4% of the program.

Overall, the activity framework provides a mix of structured regular activities combined with a dynamic community-driven curriculum. Moreover, it offers a networking venue for the TEL research community. Instructors value the opportunity to disseminate research results, promote publications and projects (and write new ones), and share knowledge. Ph.D. candidates value networking with peers at the informal learning sessions and between the sessions. In the past three years, 70–77% of Ph.D. candidates named “Discovering topics of other Ph.D. candidates” among the top three most beneficial aspects of the event in terms of learning, followed by “learning about TEL state of the art” from keynotes and thematic workshops (55–66%). In the past 3 years, 89–100% of Ph.D. candidates named “Networking with other Ph.D. candidates” among the top three benefits in terms of professional development, followed by “Networking with TEL experts” (74–78%).

Shifting of Program Topics Over Time

To investigate how topics shift in the program, we expert coded (two experts, mediating agreement) all thematic workshops and keynote sessions from the past 15 years. Each session could have multiple codes. Coding was performed inductively, starting with the first session in the first year, and adding new codes (or extending existing ones) as we went along in chronological order. The resulting matrix is sparse, and therefore was tabulated by years in order to allow for cluster analysis. The full distribution of codes over the 15 years is provided in the Supplementary Material.

We excluded topics that occurred only up to three times, based on the assumption that their low appearance frequency will inevitably lead to artifacts of a cluster analysis. We converted the tabulated data for topics by years to distances, testing Jacquard distance against binary distance measures, and testing clustering structure by inspecting the agglomerative coefficients for average, single, complete, and ward clustering methods for agglomerative nesting (agnes, package “cluster,” Maechler et al., 2019). Binary distances and Ward’s method came out top. Depending on granularity aimed for, the cluster prediction measures we consulted (Charrad et al., 2014; Kassambara and Mundt, 2019) favored – after the initial peak of 2 or 3 clusters for overview – between 6 and 8 (gap statistics), 4 and 8 (average silhouette), 6 and 8 (second differences Dindex). We ran multiple combinations, inspecting the homogeneity of the clusters via their dendrogram height, and settled on six clusters as a useful level of analysis. For the full cluster dendrogram, see the second plot in the Supplementary Material. The resulting clusters were labeled in agreement by the two human analysts (Table 2).

TABLE 2

Table 2. Clusters of topics from the thematic workshops and keynotes.

Overall, the topics of the thematic workshops change year after year, and – by their interactive nature, enforced particularly in recent years – they are more catalysts to community building than knowledge exchange. In the end, you cannot teach something that has not been invented yet. Below, we first describe each cluster on its own, and then show how it is possible to distinguish the clusters along two axes, from personal to organizational, and from knowledge to behavior in a cluster plot (Figure 3); and how the topics have developed over time (Figure 4).

FIGURE 3

Figure 3. Cluster plot of the session topics.

FIGURE 4

Figure 4. Shift of session topics over the years (Uncategorized comprises those sessions that were coded with codes that appeared overall only up to three times).

Cluster 1 – Personalized learning makes up for 25.4% of all session codes (see Figure 4). It is the dominant topic in the decade from 2005 to 2015, less prominent though in later years. The initial focus in early years on adaptation, authoring tools, and knowledge representation is extended with personal learning environments and contextualized learning, extending approaches with a behavioral (associationist) perspective. Self-regulated learning replaces the debate around informal learning.

Cluster 2 – Pervasive learning contains 36.0% of all session codes (see Figure 4). The topics in the cluster appear most often in the summer school programs, but change character over time. While early years focus on mobile learning, middle years emphasize the social character of learning, acknowledging the connectivist perspective, consequently adding learning analytics as a strong subtopic around 2012. From 2010, game-based and immersive learning became very popular themes at the summer school.

Cluster 3 – Organizational Learning contains 4.5% of all session codes (see Figure 4). In this cluster, Knowledge Maturing replaces Knowledge Work Management from early years, adding a new approach. In both cases, the focus on knowledge is complemented with a behavior perspective, looking at management of the people producing knowledge. Consequently both topics are located almost half way both from the knowledge axis extreme and its behavior counterpart. Responsiveness as a new principle is added in middle years, leveraging engagement and emotion/affect to particularly support professional contexts.

Cluster 4 – Learning environments contains 15.7% of all session codes (see Figure 4). The theme was very popular from 2005 to 2009. It started to decline in 2010 and almost disappeared as a topic, even after meta-data information extraction and learning object repositories naturally led to recommender systems, and natural language processing to technology-enhanced assessment. Human resource management as a topic was taken over in early years by Learning Processes.

Cluster 5 – Wearable Enhanced Learning contains 9.2% of all session codes (see Figure 4). A new focus on wearable enhanced learning (using accessories, headworn devices, and smart garments with embedded sensors) and on multimodal learning emerges in the last few years as a cluster on its own. The agglomerative clustering merges this with the added focus on massive open online courses (MOOCs) and gamification/badges growing from 2011. Both reflect the renewed interest to observe learner behavior beyond the cognitive, but we disagree with the grouping of the automated analysis and argue, also from the positions in the cluster plot (Figure 3), that MOOC and Gamification/Badges should be grouped together with Cluster 6, Open Education. We could imagine that wearable enhancing learning and multimodal learning/LA could additionally be grouped together with immersive technologies. Future years will reveal how the cluster structure changes, and – in our view – undoubtedly remediate the clustering artifact.

Cluster 6 – Open Education contains 4.5% of all session codes (see Figure 4). Open Content and, later, Open Data focus on Open Education for all. Recent years add a renewed focus on the use of artificial intelligence (Alexa/Siri/Cortana like learning assistants). As mentioned above, we propose to move MOOCs and Gamification/Badges into this cluster, as they are closer to the Open Education theme.

Conclusion and Discussion

We established that technology enhanced learning (TEL) is a complex field with a plethora of perspectives that benefits from disciplinary dynamics. A major challenge in advancing the field is therefore to provide suitable community spaces in which these dynamics can unfold. It is necessary for researchers to have epistemic fluency and understand sufficiently the field in order to profoundly contribute to these dynamics, while at the same time contributing rigorously to the state of the art. On this background, establishing a viable frame of reference for doctoral training in TEL is a key puzzle piece, required to drive forward the commodification of the field. Such a frame of reference is supposed to secure the interdisciplinary common ground within the field for the next generation researchers, while building shared understanding among the already established researchers.

Up to now, this frame of reference does not exist; and we perceive that the integration of interdisciplinary knowledge and ways of knowing is still ongoing. We found that this is reflected particularly in the way TEL doctoral training is organized on the institutional level, i.e., on the level of universities; answering our first research question (RQ1). Our non-exhaustive study provides evidence that TEL doctoral training is fragmented: While some next generation researchers receive interdisciplinary training already in their home institution or via the cross-institutional doctoral school, others remain trained in a monodisciplinary way. We believe that to overcome this fragmentation, the prerogative must be to not only connect isolated Ph.D. students better, but also their supervisors, directors of studies, and institutions. It requires community efforts to build and sustain professional social connections. Conferences, workshops and symposia are the traditional networking events. Existing networks, however, and the existing review criteria pose barriers for early-stage researchers for building their own social network. Doctoral summer schools and doctoral consortia are established instruments and effective tools to remediate that and support next generation researchers in developing their own social and intellectual links within the TEL community, even before publishable results are available. Both formats allow early stage as well as senior members of the community to connect over their work and sustain a continuous discourse.

Over the past 15 years, the summer school was the catalyst for moderating this perpetual discussion about the core and emerging topics, explicitly reflecting the interdisciplinary foundations of the field. Our insight from these past summer schools is that maintaining this dialogue community-driven and bottom-up is possible. The summer school provides a clear structure using an overarching activity framework to integrate the organically grown thematic structure into a complex learning experience. Despite the flexibility of this framework, it does not offer an a priori definition of basic and elective subjects (yet). Therefore, we propose that the identified structure could serve as input to a further refined cross-institutional curriculum. This structure requires expert agreement found in curricular commissions involving all key stakeholders in order to clarify the mandatory and optional elements for training researchers in the field of TEL. Such an offer can help institutions to overcome potential local shortcomings, while preserving the bottom-up prevalent community-driven, culture. This answers our second research question (RQ2) about the need and characteristics of complementary cross-institutional training activity for TEL doctoral-level research.

Beyond cross-institutional doctoral education activities, we identified the scarcity of shared educational resources as a key gap in current TEL doctoral training. Earlier efforts regarding TEL OER do exist, resulting in a TEL dictionary; and a collection of educational resources at doctoral level, albeit with a stronger emphasis on general learning sciences than having a TEL-specific focus. These existing efforts need to be updated and extended for adapting to latest developments of a constantly changing field. OER are useful tools for doctoral education beyond their educational use, for example by involving early stage researchers as authors in the participatory development of resources and concepts. This aims at lowering the barriers for early stage researchers to leave their mark in our interdisciplinary community, by codifying and preserving the established common ground. In parallel to creating a stronger base in OERs, the element of openness can be extended toward open science in the broader sense. The TEL field needs more open data gathering, curation, sharing, and re-use activities that strengthen evidence-based research, while complying with data protection regulations. Doctoral education is the perfect place to promote and discuss the practices of open science within TEL.

Finally, beyond TEL doctoral training, strategic and integrative activities exist of course that contribute to bringing TEL forward as an interdisciplinary field by bridging across communities. Examples are roadmapping and observatory initiatives like, e.g., the “Innovating Pedagogy” reports of the Open University of the United Kingdom, the “Horizon Reports” formerly of the New Media Consortium, now as part of Educause, or the “Emerging state of XR and Immersive Learning” report of the Immersive Learning Research Network. Complementing such observatory activities as integrative across institutions and communities with overlapping interests, a liaison across a number of scientific societies in TEL and closely related fields, the International Alliance to Advance Learning in the Digital Era (IAALDE), has been set-up, which fosters cross-fertilization across a broad range of research communities by exchanging best papers between conferences.

This study was limited several ways. First, the review of scientometric meta-analysis of the field is dated and more of historic value, shedding light on the foundation of TEL. We hope that with this article, we contribute to a long-overdue update, but we also have to acknowledge that the scope of our analysis was rather limited, sacrificing breadth (TEL community at large) for depth (TEL doctoral training). Moreover, we limited our analysis of educational doctoral training programs geographically to Europe. To overcome these limitations, we propose to study in more depth, also in quantitative ways, the existing TEL doctoral training globally, including ways of promoting cross-institutional structures and resources. Future investigations should also help account for career paths of TEL alumni as academics, EdTech entrepreneurs, or executives in the knowledge-oriented economy. Additionally, further analysis as to how to teach students with diverse educational backgrounds and how to overcome inevitable problems would serve useful to the field.

This would not only help to document another snapshot of the field with a focus on doctoral education. It would additionally serve to inform professional societies, Higher Education Institutions, as well as beneficiaries of TEL alike about the composition of the field, the current state of the art, and about the human talent available. Such stock-taking would ensure the world is better prepared for lock-down imposed by a pandemic such as COVID-19, where technology enhanced learning is the only viable option for education and training at large. Ultimately, this could also help TEL research to have a higher impact on curricula in teacher training: In the end, TEL research is essential in supporting policy makers with the ambitious goal of the digitization of society.

Data Availability Statement

The data analyzed in this study are subject to the following licenses/restrictions: The data are partially available in the article/Supplementary Material, in particular, the full list of all summer school sessions is available on request. Requests to access these datasets should be directed to mikhail.fominykh@ntnu.no.

Author Contributions

VP-S contributed to lead manuscript-writing, overall edits, including structure and main line of argumentation, and discussions. FW contributed to overall manuscript-writing, manuscript strategy, including structure and main line of argumentation, discussions, and summer school analysis. MF contributed to overall manuscript-writing and 15 years of EATEL summer school analysis. TL contributed to overall manuscript-writing and discussions, and focus on TEL as an interdiscipline and examples. MP contributed to coordination of the research/writing of section “Methodology” TEL Doctoral Training in European Universities. MS contributed to discussions, focus on section “Methodology” TEL Doctoral Training in European Universities. DH-L contributed to sections “Introduction”, “Technology Enhanced Learning as an Interdiscipline”, and “Conclusion”. MK contributed to discussions, focus on definitions of interdisciplinarity, and meta-analysis of the dynamics of the field. RK contributed to discussions and meta-analysis of the dynamics of the field. LP and CS contributed to describing the relevance of doctoral training. CG and AP contributed to doctoral education in Europe in TEL. AE contributed to discussions and description of section “Methodology” TEL Doctoral Training in European Universities. EP-F contributed to discussion. DG and KM: contributed to 15 years of EATEL summer school analysis. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the European Commission under the Erasmus+ Program, as part of the DE-TEL project (Grant Agreement No. 2019-1-NO01-KA203-060280) and under the Horizon 2020 Program, as part of the ARETE project (Grant Agreement No. 856533).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2020.00150/full#supplementary-material

Footnotes

1. ^ www.ea-tel.eu

References

Aboelela, S. W., Larson, E., Bakken, S., Carrasquillo, O., Formicola, A., Glied, S. A., et al. (2007). Defining interdisciplinary research: conclusions from a critical review of the literature. Health Serv. Res. 42, 329–346. doi: 10.1111/j.1475-6773.2006.00621.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Amran, N. N., and Ibrahim, R. (2012). Academic rites of passage: reflection on a PhD journey. Procedia Soc. Behav. Sci. 59, 528–534. doi: 10.1016/j.sbspro.2012.09.310

CrossRef Full Text | Google Scholar

Buchem, I., Klamma, R., and Wild, F. (2019). “Introduction to wearable enhanced learning (WELL): trends, opportunities, and challenges,” in Perspectives on Wearable Enhanced Learning (WELL): Current Trends, Research, and Practice, eds I. Buchem, R. Klamma, and F. Wild (Cham: Springer).

Google Scholar

Carr, G., Loucks, D., and Blösch, G. (2018). Gaining insight into interdisciplinary research and education programmes: a framework for evaluation. Res. Policy 47, 35–48. doi: 10.1016/j.respol.2017.09.010

CrossRef Full Text | Google Scholar

Charrad, M., Ghazzali, N., Boiteau, V., and Niknafs, A. (2014). NbClust: an R package for determining the relevant number of clusters in a data set. J. Stat. Softw. 61, 1–36.

Google Scholar

Cleverdon, C. W. (1960). The aslib cranfield research project on the comparative efficiency of indexing systems. ASLIB Proc. 12, 421–431. doi: 10.1108/eb049778

CrossRef Full Text | Google Scholar

Derntl, M., and Klamma, R. (2012). “The European TEL projects community from a social network analysis perspective,” in Proceedings of the 21st Century Learning for 21st Century Skills. 7th European Conference on Technology Enhanced Learning, EC-TEL 2012, eds A. Ravenscroft, S. Lindstaedt, C. D. Kloos, and D. Hernández-Leo (Berlin: Springer Verlag), 51–64. doi: 10.1007/978-3-642-33263-0_5

CrossRef Full Text | Google Scholar

Fessl, A., Wesiak, G., Rivera-Pelayo, V., Feyertag, S., and Pammer, V. (2017). In-app reflection guidance: lessons learned across four field trials at the workplace. IEEE Trans. Learn.Technol. 1, 488–501. doi: 10.1109/tlt.2017.2708097

CrossRef Full Text | Google Scholar

Hall, S., Wild, F., and Olde Scheper, T. (2019). “Real-time auditory biofeedback system for learning a novel arm trajectory: a usability study,” in Perspectives on Wearable Enhanced Learning (WELL) Current Trends, Research, and Practice, eds I. Buchem, R. Klamma, and F. Wild (Cham: Springer).

Google Scholar

Herodotou, C., Villasclaras-Fernández, E., and Sharples, M. (2014). “The design and evaluation of a sensor-based mobile application for citizen inquiry science investigations,” in Open Learning and Teaching in Educational Communities. EC-TEL 2014. Lecture Notes in Computer Science, Vol. 8719, eds C. Rensing, S. de Freitas, T. Ley, and P. J. Muñoz-Merino (Cham: Springer).

Google Scholar

ISCED (2011). International Standard Classification of Education. Paris: UNESCO Institute for Statistics.

Google Scholar

Kalz, M., and Specht, M. (2014). Assessing the cross-disciplinarity of technology-enhanced learning with science overlay maps and diversity measures. Br. J. Educ. Technol. 45, 415–427. doi: 10.1111/bjet.12092

CrossRef Full Text | Google Scholar

Kassambara, A., and Mundt, F. (2019). factoextra: Extract and Visualize the Results of Multivariate Data Analyses. R Package Version 1.0.6.

Google Scholar

Kittur, A., Chi, E. D. H., and Suh, B. (2008). “Crowdsourcing user studies with mechanical turk,” in Proceedings of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems, New York, NY: ACM, 453–456.

Google Scholar

Kuhn, T. S. (2012). The Structure of Scientific Revolutions. 50th Anniversary Edition. Chicago, IL: University of Chicago press.

Google Scholar

Latour, B. (2005). Reassembling the social. An introduction to actor-network-theory. Oxford: Oxford University Press.

Google Scholar

Lave, J., and Wenger, E. (1995). Situated Learning - Legitimate Peripheral Participation. Cambridge: Cambridge University Press.

Google Scholar

Ley, T., Maier, R., Thalmann, S., Waizenegger, L., Pata, K., and Ruiz-Calleja, A. (2019). A knowledge appropriation model to connect scaffolded learning and knowledge maturation in workplace learning settings. Vocat. Learn. 13, 91–112. doi: 10.1007/s12186-019-09231-2

CrossRef Full Text | Google Scholar

Limbu, B., Vovk, A., Jarodzka, H., Klemke, R., Wild, F., and Specht, M. (2019). “WEKIT.one: a sensor-based augmented reality system for experience capture and re-enactment,” in Transforming Learning with Meaningful Technologies, eds M. Scheffel, J. Broisin, V. Pammer-Schindler, A. Ioannou, and J. Schneider (Cham: Springer).

Google Scholar

Lindvig, K. (2018). The implied PhD student of interdisciplinary research projects within monodisciplinary structures. Higher Educ. Res. Dev. 6, 1171–1185. doi: 10.1080/07294360.2018.1474343

CrossRef Full Text | Google Scholar

Maechler, M., Rousseeuw, P., Struyf, A., Hubert, M., and Hornik, K. (2019). cluster: Cluster Analysis Basics and Extensions. R Package Version 2.1.0.

Google Scholar

Maier, R., and Schmidt, A. (2015). Explaining organizational knowledge creation with a knowledge maturing model. Knowl. Manag. Res. Pract. 13, 361–381. doi: 10.1057/kmrp.2013.56

CrossRef Full Text | Google Scholar

Markauskaite, L., and Goodyear, P. (2017). Epistemic Fluency and Professional Education. Dordrecht: Springer.

Google Scholar

Meyer, P. (2011). Technology-Enhanced Learning’ as an Interdisciplinary Epistemic Community. master thesis, University of Augsburg, Augsburg.

Google Scholar

Meyer, P., Kelle, S., Ullmann, T. D., Scott, P., and Wild, F. (2013). “Interdisciplinary cohesion of TEL – an account of multiple perspectives,” in Scaling up Learning for Sustained Impact. EC-TEL 2013. Lecture Notes in Computer Science, Vol. 8095, eds D. Hernández-Leo, T. Ley, R. Klamma, and A. Harrer (Berlin: Springer).

Google Scholar

O’Rourke, M., Crowley, S., and Gonnerman, C. (2016). On the nature of cross-disciplinary integration: a philosophical framework. Stud. History Philos. Biol. Biomed. Sci. 56, 62–70. doi: 10.1016/j.shpsc.2015.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Pham, M. C., Derntl, M., and Klamma, R. (2012). Development patterns of scientific communities in technology enhanced learning. Educ. Technol. Soc. 15, 323–335.

Google Scholar

Ritter, S., Anderson, J. R., Koedinger, K. R., and Corbett, A. (2007). Cognitive tutor: applied research in mathematics education. Psychon. Bul. Rev. 14, 249–255. doi: 10.3758/BF03194060

PubMed Abstract | CrossRef Full Text | Google Scholar

Rodríguez-Triana, M. J., Prieto, L. P., Ley, T., de Jong, T., and Gillet, D. (2019). “Tracing teacher collaborative learning and innovation adoption : a case study in an inquiry learning platform,” in Proceedings of the CSCL 2019 - International Conference of Computer Supported Collaborative Learning, France, doi: 10.22318/cscl2019.432

CrossRef Full Text | Google Scholar

Seitlinger, P., Bibi, A., Uus, Õ, and Ley, T. (2020). “How working memory capacity limits success in self-directed learning: a cognitive model of search and concept formation,” in Proceedings of the 10th International Conference on Learning Analytics & Knowledge, New York, NY: AMC.

Google Scholar

Shneiderman, B. (2008). Science 2.0. Science 319, 1349–1350.

Google Scholar

Tchounikine, P. (2011). Computer Science and Educational Software Design. Berlin: Springer.

Google Scholar

Van Merriënboer, J. J. G. (2019). ). The Four-Component Instructional Design Model: An Overview of its Main Design Principles. Maastricht: Maastricht University.

Google Scholar

Wagner, C., Roessner, J., Bobb, K., Thompson Klein, J., Boyack, K. W., Keyton, J., et al. (2011). Approaches to understanding and measuring interdisciplinary scientific research (IDR): a review of the literature. J. Infometr. 5, 14–26. doi: 10.1016/j.joi.2010.06.004

CrossRef Full Text | Google Scholar

Wild, F. (2016). Learning Analytics in R. Berlin: Springer.

Google Scholar