Scale to measure student perception in collaborative online international learning experiences: design and validation

Introduction: Digital education is favorably positioned as a learning option; it employs various strategies, pedagogies, and methodologies, including collaborative online international learning experiences (COIL). This alternative, inherent to the digital era, transcends traditional educational methods by boosting technology to enhance learning experiences. Its given purpose is to improve learning gains, foster interculturality, internationalizing the curriculum, and strengthening skills necessary for the 21st century. Moreover, it can potentially address global educational needs that have not been fulfilled in the past.

Methods: This study conducted a content validation of a scale (instrument) intending to measure students' perception of collaborative online international learning experiences (COIL) in three dimensions: (a) interaction among students in digital environments, (b) collaborative work in multicultural teams to achieve goals, and (c) peer reflection on differences and similarities during collaboration. The study employed the Delphi method of expert judgment.

Results: In the overall scale, Aiken's V values indicated that the clarity criterion did not attain an acceptable score. Therefore, a review is desired to determine which instrument items need reformulation. However, Aiken's V scores met acceptable coherence, relevance, and sufficiency values.

Conclusion: The proposed scale contributes to research on collaborative online international learning experiences, serving as a valuable tool for future investigations, particularly those focused on measurement, and as a reference for evaluating COIL experiences among students.

1 Introduction

Formal and informal online education is becoming increasingly common due to the diverse options available on the internet and the efforts governments are making to achieve the fourth Sustainable Development Goal: quality education. Focusing on formal higher education, governments act significantly to ensure quality and widespread education in their countries. This is evidenced by the fact that from 1970 to 2020, enrollment in higher education institutions increased yearly worldwide (World bank, 2024). Confirming this, Mok and Marginson (2021) also reported that East Asia's enrollment attained over 50 million students in 2018, consistent with the global acceleration and the trends of massification, diversification, and internationalization. These figures do not distinguish learning modalities (face-to-face, distance, blended, or virtual) (Imran et al., 2023). However, focusing on non-formal online education, it can be affirmed that what (Noam, 1995) predicted in the 1990s has come true: with e-learning, universities would not be the protagonists, but commercial enterprises would provide sophisticated courses on the web. The data show that many higher education institutions have created virtual or online programs, extending coverage to previously unattainable contexts. The online programs intend to create memorable digital experiences for each learner through active, student-centered methodologies.

Although there are still barriers and gaps in internet access, the proliferation of platforms offering courses, micro-courses, certifications, Massive Open Online Courses (MOOCs), and academic information through various channels and media is diverse. Some are backed by academic institutions, others by the instructor's reputation, and others align with current technological trends, all advocating for educational innovation that meets the training needs of the current moment. Indeed, the effectiveness of these online learning offerings is not primarily determined by the factors mentioned earlier. Instead, the effectiveness of e-learning primarily requires collaboration and student involvement with the proposed activities and strategies (Alyoussef, 2023). Along these lines, research by Baloran et al. (2021) and Elshami et al. (2022) emphasize that student engagement is one of the main challenges of e-learning. While there is generally a high level of student satisfaction with courses developed using e-learning, the concern lies in achieving student participation in these complex learning processes. Furthermore, several studies have found that participation indicates student engagement and correlates to performance and satisfaction (Greller et al., 2017; Sinclair et al., 2017; Rajabalee et al., 2020). On the other hand, some perceptions about e-learning assert that online learning is merely the use of technologies for information dissemination (Tajik and Vahedi, 2021). The issue is that e-learning is as broad as face-to-face education; the teacher's intention to adopt strategies employing digital technologies that support and enhance planned contextual activities is critical. Other determinant elements include connectivity, accessibility, usability, and required skills.

1.1 Conceptualizing COIL

COIL experiences are a digital educational alternative that connects two or more educational institutions in a student-centered online learning environment to address, understand, or discuss a global issue of interest to the parties. Some research has labeled it a pedagogy that creates environments for developing intercultural skills and competencies using technologies connecting two classrooms in different geographical areas (Appiah-Kubi and Annan, 2020; Zapatero et al., 2022).

On the other hand, Davis et al. (2023) define COIL as an innovative pedagogical approach used worldwide, providing teachers and students with the opportunity to engage in a global learning activity where they learn from and about each other (Marcillo-Gómez and Desilus, 2016). In addition to the predominant focus on interculturality and internationalization, it is also an innovative and cost-effective pedagogical tool that offers global learning opportunities from home (Vahed and Rodriguez, 2020; Liu and Shirley, 2021). Notably, COIL experiences have different conceptions, but all are closely related to internationalization, global learning, internet use, collaboration, and interculturality.

1.2 COIL and competencies for the digital era

Up to this point, it can be observed that digital education is still under construction. Although it has been massified, ongoing inquiry into its results and implications continues. Various actors are involved in creation, production, and delivery. However, universities and higher education institutions are the foremost correspondents, as they have responded to the needs of different eras throughout history. Current education must integrate interculturality, address problems beyond the classroom, pay attention to digital nomads, innovate with tangible solutions that benefit communities and society as a whole through digital means, and meet specific demands (Tajpour and Hosseini, 2021; Hajimiri et al., 2023; Mukul and Büyüközkan, 2023). collaborative online international learning (COIL) courses address several of the abovementioned elements and serve as a strategy for active digital learning that vigorously fosters competencies in students who benefit from the dynamics of these experiences (Watla-iad and Kradtap Hartwell, 2022). This approach uses digital media to foster synchronous and asynchronous online collaboration among students and teachers in different geographical contexts. It creates equitable, innovative learning environments that align with 21st-century classrooms, enhancing cultural sensitivity and improving learning experiences (Borger, 2022). The strategy provided here is inherent to the digital age but requires further research and evidence to continue optimal and effective implementation.

COIL experiences are increasingly being implemented in universities for various reasons beyond internationalization or interculturality. They were not only used during the COVID-19 pandemic, although that situation gave them momentum at the time because it was a means to address internationalization processes without leaving home; they have been utilized for over two decades. They currently emerge as a coherent educational solution to the digital environments presented.

The nature of COIL experiences renders some competencies imminent and natural, such as interculturality, which typically involves two groups of students from educational institutions in different countries (West et al., 2022; Hackett et al., 2023). Other inherent competencies include strengthening a second or foreign language, in some cases English (Pouromid, 2019; Helena and Alena, 2021). However, other less direct competencies have been identified, such as job training (Nethsinghe et al., 2023). This competency may not be perceived in all studies or experiences, as it depends on the intention of the planned COIL. However, most COIL courses are designed in an initial phase of discussion and argumentation about a specific case, situation, or problem (Ingram et al., 2021; House et al., 2022). Some studies provide evidence of a strategy employing a real challenge or problem that promotes contextualized interdisciplinary work and requires a possible solution. In such cases, student involvement is greater, and the bond is very close; therefore, the competencies and skills developed differ. The content tends to be global and proposes complex situations (Suarez and Michalska Haduch, 2020; Salmon et al., 2022). As can be observed, studies report the benefits of developing various competencies that participants positively perceive. However, validated instruments that measure students' perception of COIL experiences in the three dimensions illustrated in Figure 1 are not found (Goto and Gutierrez-Gomez, 2024; Mestre-Segarra and Ruiz-Garrido, 2022; Wimpenny and Orsini-Jones, 2020).

Figure 1

Figure 1. Perception of COIL experiences.

1.2.1 Student interaction in digital environments

The construction of learning through social interactions allows individuals to build their knowledge based on their experiences and the use of digital media, enabling synchronous or asynchronous collaboration, communication, and creation in previously unimagined spaces (SUNY COIL CENTER, 2013; Weller et al., 2020).

1.2.2 Collaborative work in multicultural teams to achieve objectives

The classroom is conceived as a community of individuals who, regardless of their culture and language, engage collaboratively and enrich discussions and dialogues from their respective viewpoints. This facilitates understanding the topics or issues addressed with a broader and more diverse perspective, positively impacting learning outcomes and skills development (SUNY COIL CENTER, 2013).

1.2.3 Reflection on differences and similarities with peers during collaboration

This has a student-centered focus stemming from involvement and active roles in these courses, where the intercultural exchange is facilitated through activities that allow for a broader global dimension of course content, fostering reflection among all involved stakeholders (SUNY COIL CENTER, 2013).

This brief conceptualization of the dimensions suggests that COIL experiences profoundly affect students' comprehensive and generic understanding; they are the foundations upon which these courses are built. For this reason, a questionnaire-type instrument was designed with the purpose of being used in various COIL courses to effectively measure the above-described constructs. The aim was to collect data that allows for the systematization of experiences and potentially generate a theory postulating the principles upon which COIL experiences should remain grounded.

2 Method

2.1 Instrument design

For the design of the instrument, a literature review was conducted to determine which would be the appropriate dimensions to include in the scale, for this purpose, different proposals of measurement instruments developed to measure COIL experiences in higher education students were reviewed. As a result of this literature review and the three dimensions to measure COIL experiences were determined, this initial proposal of the scale had three dimensions, after the decision and the definition of the constructs, we proceeded to the design and adaptation of the items that would make up the scale. For cultural sensitivity, the internal review of the items by teachers and students who have participated in COIL experiences was taken into account, in order to determine that the items were clear to the context; after this internal review, the first version was prepared to proceed to content validation by expert judgment. The Questionnaire on the perception of the student's role in the Collaborative International Online Learning Experiences (COIL) can be consulted at: https://doi.org/10.6084/m9.figshare.26210861.v1.

2.2 Participants

For the selection of the members of the expert judgment panel, a search was carried out in Scopus for authors on the subject of COIL, the inclusion criteria being that the authors had at least 3 articles published on the subject as first author or corresponding author. After the selection of the candidates, an e-mail was sent to them inviting them to participate, for which 45 invitations were sent, however, only 18 experts participated.

The sample comprised 18 experts who validated the content of the previously designed questionnaire. In total, 11 women and seven men participated, with 12 holding a doctoral degree. The experts came from institutions in more than six countries, with the highest representation from the United States, Spain, Mexico, and Colombia. Their professional experience ranged from 6 to 32 years (see Table 1).

Table 1

Table 1. Demographic information of the expert judgment participants.

2.3 Procedure

For the validation of the content, the Delphi method was used, which is a technique to achieve consensus among a group of people who are part of a panel of experts. This method organizes the communication process of a panel of experts focused on evaluating a problem in several rounds, in this case evaluating the design of a measurement instrument (García-Valdés and Suárez-Marín, 2013; López-Gómez, 2018). The intention of the Delphi methodology is to collate individual opinions until they reach a statistically generated consensus with collective intelligence (Nasa et al., 2021). In this case, for the validation of the content, evaluation criteria were used through which the experts should give their judgment on the items that made up the measurement scale, aspects such as clarity, coherence, relevance and sufficiency were considered. A matrix was developed which defined the elements to be evaluated and the values to be placed, the values ranged from 1 (“criteria not met”) to 4 (“high level”) (see Table 2).

Table 2

Table 2. Reference values for expert judgment evaluation criteria.

The expert judgment process consisted of two stages: the first involved responding to questions related to professional variables, and the second involved evaluating the items comprising the instrument, applying the established evaluation criteria. Lastly, experts provided comments and suggestions for improvement for each item (Escobar-Pérez and Cuervo-Martínez, 2008). For the analyses, the measures of central tendency and normality of the data were taken into account. With respect to the normality of the data, acceptable values of skewness and kurtosis, which were within 2 and −2 standard deviations, were taken as reference (George and Mallery, 2001). Regarding the values obtained with Aiken's V coefficient, values above 0.75 were considered as criteria for permanence (Wilcox and Serang, 2017; Aw, 2019). Items with values below this threshold were considered to need further review.

3 Results

3.1 The criterion of clarity in the wording of the scale items

Table 3 shows the results of the measures of central tendency, normality, and Aiken's V scores for the clarity criterion for each item of the International Online Collaborative Learning Perception Scale (COIL) assessed by the expert judges. The means indicate that experts considered the items to have a moderate level of clarity in their wording, except items 6 and 9, whose means suggested unclear wording. Regarding the distribution of the data, with the exception of items 3, 11, 14, 16 and 18, the rest presented normal values considered acceptable.

Table 3

Table 3. Central tendency, normality, and Aiken's V calculated for item clarity in the Collaborative Online International Learning (COIL) scale.

Regarding content validity through expert judgment, reference values were used to decide whether items should be retained based on items with an Aiken's V above 0.75. Items that did not meet this criterion should be reassessed for necessary adjustments in their wording. The obtained values ranged between 0.50 and 0.87 points. Eleven of the 19 items comprising the scale were considered clear in their wording and could be considered for measuring the construct. However, eight (items 1, 5, 6, 8, 9, 12, 16, 19) had values below the established reference value, indicating poor clarity of wording. Therefore, these items needed to be adjusted and reevaluated by expert judgment.

3.2 The criterion of coherence in the wording of the scale items

Table 4 reports the measures of central tendency, normality, and Aiken's V scores for the items' coherence criterion. The scores reflect that the experts considered the items coherent for what they intended to measure. Regarding normal distribution, the items exhibited skewness and kurtosis values within acceptable ranges, except for items 2, 3, 4, 8, 10 and 11, which exceeded acceptable normality values. Concerning content validity, Aiken's V values show that of the 19 items comprising the scale, 17 attained values considered acceptable (>0.75). However, two (items 9 and 17) need to be reviewed for coherence in the construct.

Table 4

Table 4. Central tendency, normality, and Aiken's V calculated for item coherence in the Collaborative Online International Learning (COIL) scale.

3.3 The criterion of relevance in the wording of the scale items

Table 5 reports the measures of central tendency, normality, and Aiken's V scores for the items' relevance criterion. The scores reflect that experts considered the items relevant for the intended measurement. Regarding the distribution of the data, with the exception of items 1, 2, 3, 4, 5, 6, 7, 8, 8, 10, 11, 13, 14, 16 and 18, the rest presented values of normality considered acceptable. Concerning content validity, Aiken's V values indicated that 18 of the 19 items comprising the scale were acceptable (>0.75). However, one item (item 9) must be reviewed to assess its relevance for inclusion in the Perception of collaborative online international learning (COIL) experiences scale.

Table 5

Table 5. Central tendency, normality, and Aiken's V calculated for item relevance in the Collaborative Online International Learning (COIL) scale.

Table 6 reports Aiken's V values for the clarity, coherence, relevance, and sufficiency criteria for the three dimensions comprising the scale and the overall scale. In Dimension 1, which corresponds to the interaction among students in digital environments, it was found that the clarity of item wording and sufficiency of items should be addressed by the authors who designed the scale to attain acceptable values and ensure that these criteria are satisfactorily met. In the second dimension, referring to collaborative work in multicultural teams to achieve objectives, it was found that the clarity criterion for item wording did not meet the minimum required value; however, the coherence, relevance, and sufficiency criteria achieved values above the suggested threshold. The third dimension of the instrument, concerning reflection on differences and similarities among peers during collaboration, met all criteria assessed by the experts, meaning that the items comprising this dimension were clear, coherent, relevant, and sufficient for the intended measurement. Finally, in the overall score of the scale, Aiken's V values indicated that the clarity criterion did not attain the acceptable score, prompting a review to identify which items need to be reformulated. However, in terms of coherence, relevance, and sufficiency, the overall Aiken's V scores had acceptable values.

Table 6

Table 6. Aiken's V values for the clarity, coherence, relevance, and sufficiency criteria reported for the three dimensions comprising the scale and for the overall scale.

4 Discussion

Measuring international online collaborative learning is essential to understanding and improving the effectiveness of these global educational experiences. This study aimed to assess the content validity through the Delphi method of the instrument that measures the perception of Collaborative International Online Learning (COIL) experiences. For this purpose, the criteria of clarity, coherence, relevance and sufficiency were considered.

The results obtained from the evaluation of the criteria of clarity, coherence, relevance and sufficiency of the collaborative online international learning (COIL) instrument showed important aspects to be considered in the design of the measurement scale. In the evaluation of the criterion of clarity by items of the instrument, it was found to be weak, more than half of the items did not achieve acceptable values of Aikeen's V (Ventura-León, 2019). In the criterion of item coherence, values were reported that allow assuming that the items are suitable for measuring the construct, with the exception of two items that need to be evaluated to ensure that this criterion is effectively met and thus guarantee their coherence and inclusion in the measurement scale. Regarding the relevance of the items, Aikeen's V values were considered acceptable, which guarantees that the items designed are relevant to measure the construct, with the exception of only one item that did not meet the suggested values.

These findings show that although the criteria of consistency and relevance of the items can be considered adequate, the clarity of the items should be reevaluated to make the necessary adjustments. One possible cause of the lack of clarity in the items is the absence of a consensus on the definition of COIL. This also affects the operationalization of the dimensions that should be included to measure the construct. Although there are a few studies on this topic, they report different evidences of validity. However, none of these studies consider the dimensions included in this instrument (Deardorff, 2006; Razali et al., 2016; Biasutti and Frate, 2018; Shimizu et al., 2020; Palacios-Núñez et al., 2023). It should be clarified that COIL has been used for different educational purposes, such as fostering motivation, strengthening or developing internationalization and multiculturalism, favoring different competencies, positively impacting learning, or addressing global issues, which makes it difficult to measure. For this research, we start from what has been promoted by the State University of New York (SUNY) and its creators (Rubin and Guth, 2022), based on three elements that this methodology promotes, which are: collaboration, interaction and intercultural exchange. Each of the constructs could be measured separately or some could be added according to interest or purpose, but the value of COIL is based on the convergence of the triad that sustains them.

It is key to mention that this type of instrument is necessary for institutions and teachers to be able to measure with certainty the effectiveness of the efforts pursued in this type of COIL experiences. This makes it possible to continue building more robust experiences and to obtain a more holistic understanding of online learning environments and the constructs on which they are based. In the case of measuring a specific construct, it is recommended to use instruments or models designed for this purpose, such as intercultural competence (Deardorff, 2006; Hofhuis et al., 2020) or collaboration and interaction related to learning (Collazos et al., 2007), just to cite a few examples. But if what is intended is to measure the perception of students in a COIL experience in a general way, this instrument is a relevant and useful tool for this purpose.

In addition to the evaluation of the assessment by items, they were also analyzed at the dimension and global score level, in this part of the analysis the sufficiency dimension of the items was included. This made it possible to know which of the three dimensions that make up the instrument: interaction among students in digital environments, collaborative work in multicultural teams to achieve objectives, and reflection on differences and similarities with peers during collaboration present values that imply a restructuring of the scale. The results of the scores in the dimension of interaction between students in digital environments need to be subjected to a second evaluation to determine the clarity of the wording of the items and sufficiency, since the minimum scores in the values of Aiken's V were not reached.

The Aiken V values of the dimension of collaborative work in multicultural teams for the achievement of objectives indicate that the criterion of clarity should be addressed again to verify those items that do not reach the desired scores and to ensure that the items are precise in their wording; however, the other criteria, coherence, relevance and sufficiency meet acceptable values. Finally, the third dimension, referring to reflection on differences and similarities with peers during collaboration, meets all the criteria evaluated by the experts. This reflects that the items that make up this dimension are clear, coherent, relevant and sufficient to measure the construct. The results obtained by dimensions show that the weakest criterion is clarity, which implies that the items that make up the scale should be reviewed and the wording rethought.

Regarding the overall scale scores, once again, the authors need to address the criterion of clarity in the wording and readjust the items that did not reach the minimum acceptable scores per item, which compromised the scores per dimension and the overall scale. However, a positive aspect is the acceptable values in the other criteria (coherence, relevance and sufficiency), which guarantee the content validity of the instrument measuring collaborative online international learning (COIL) experiences.

5 Conclusions

In this study, expert judges evaluated the content validity of an instrument designed to measure the perception of International Collaborative Online Learning Experiences (COIL) for three proposed dimensions using the criteria of clarity, coherence, relevance and sufficiency. The need to review the clarity of certain items was identified, as eight of the nineteen items did not reach the suggested values for retention on the scale, implying a possible need for reformulation to improve comprehension. Although some items required adjustments in coherence and relevance, the majority of the scale items demonstrated acceptable values for these criteria, supporting its content validity in measuring perceptions of COIL experiences. Despite the challenges identified, this study provides a valuable tool for future research on international online collaborative learning.

5.1 Limitations and future research

This study contributes to research on collaborative international online learning experiences, specifically in the field of measurement. Therefore, this study can serve as a reference for evaluating students' COIL experiences. However, the results need to be approached with caution due to the inherent limitations of this study. For example, the evaluation of expert judgment was based on the opinion of a specific group, which could limit the generalizability of the results. A larger sample of experts could provide a more robust validation of the instrument. Further rounds among the experts are also needed to achieve consensus on the dimensions assessed. In addition, although this study presents a comprehensive assessment of content validity, it is necessary to evaluate other validity evidence such as construct validity, convergent and divergent validity, assessment of measurement invariance, and latent mean analysis that could provide information about the behavior of the data in the design. measurement model.

For future research, studies should be conducted that address the measurement of collaborative international online learning experiences, focusing on understanding the COIL construct through the proposed dimensions. In addition, it is essential to develop and validate specific instruments that focus on critical aspects of COIL contexts, such as intercultural communication and conflict resolution. Finally, longitudinal research is needed to provide a deeper understanding of the evolution of students' perceptions over time and the influence of COIL on students' academic performance and satisfaction.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

HR-G: Conceptualization, Investigation, Writing – original draft, Writing – review & editing. CA-N: Investigation, Methodology, Supervision, Writing – original draft, Writing – review & editing. MR-M: Investigation, Methodology, Project administration, Validation, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. The authors appreciate the funding from the Grant DigiUGov ERASMUS-2022-CBHE and “Challenge-Based Research Funding Program 2022,” Project ID # I005 - IFE001 - C2-T3 – T.

Acknowledgments

The authors acknowledge the financial and technical support of Writing Lab, Institute for the Future of Education, Tecnologico de Monterrey, Mexico, in the production of this work. In addition, the authors thank and value the academic contribution of each of the expert judges: Amy Anderson (Spokane Community College), Melissa Saftner (University of Minnesota), José María Romero Rodríguez (University of Granada), Alberto San Martín Zapatero (Universidad de Burgos. España), Patricia Munoz de Escalona (Glasgow Caledonian University), Daniel Rodríguez Marconi (Universidad Católica de Temuco), Felipe Furtado Guimaraes (Universidade Federal do Espírito Santo), Maria Ballesteros-Sola (California State University Channel Islands), Linda Sue Hammonds (University of South Alabama), Jonah M. Otto (University of Augsburg), Rebeca María García García (Tecnológico de Monterrey), Leonardo David Glasserman Morales (Tecnológico de Monterrey), Julio César Ramírez Montañez (Universidad Pontificia Bolivariana), Luisa María Lozano Díaz (Universidad de La Sabana), Angélica Milena Ramírez Bustamante (Universidad de La Sabana), y Leidy Evelyn Díaz Posada (Universidad de La Sabana).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Alyoussef, I. Y. (2023). Acceptance of e-learning in higher education: the role of task-technology fit with the information systems success model. Heliyon 9:e13751. doi: 10.1016/j.heliyon.2023.e13751

PubMed Abstract | Crossref Full Text | Google Scholar

Appiah-Kubi, P., and Annan, E. (2020). A review of a collaborative online international learning. Int. J. Eng. Pedagog 10, 109–124. doi: 10.3991/ijep.v10i1.11678

Crossref Full Text | Google Scholar

Aw, S. (2019). Developing an evaluation instrument for assessing public relations practitioner performance in educational institutions. Malays. Online J. Educ. Manag. 7, 20–36. doi: 10.22452/mojem.vol7no1.2

PubMed Abstract | Crossref Full Text | Google Scholar

Baloran, E. T., Hernan, J. T., and Taoy, J. S. (2021). Course satisfaction and student engagement in online learning amid COVID-19 pandemic: a structural equation model. Turk. Online J. Distance Educ. 22, 1–12. doi: 10.17718/tojde.1002721

Crossref Full Text | Google Scholar

Biasutti, M., and Frate, S. (2018). Group metacognition in online collaborative learning: validity and reliability of the group metacognition scale (GMS). Educ. Technol. Res. Dev. 66, 1321–1338. doi: 10.1007/s11423-018-9583-0

Crossref Full Text | Google Scholar

Borger, J. G. (2022). Getting to the core of collaborative online international learning (COIL). Front. Educ. 7:987289. doi: 10.3389/feduc.2022.987289

Crossref Full Text | Google Scholar

Collazos, C., Renzi, S., Guerrero, L., Pino, J., Ortega, M., Bravo, C., et al. (2007). Evaluating collaborative learning processes using system-based measurement. J. Educ. Technol. Soc. 10, 257–274. Available at: https://repositorio.uchile.cl/handle/2250/125886

Google Scholar

Davis, L. L., Bhatarasakoon, P., Chaiard, J., Walters, E. M., Nance, J., Mittal, M., et al. (2023). Use of collaborative online international learning to teach evidence-based practice. J. Nurse Practit. 19:104498. doi: 10.1016/j.nurpra.2022.11.008

Crossref Full Text | Google Scholar

Deardorff, D. K. (2006). Identification and assessment of intercultural competence as a student outcome of internationalization. J. Stud. Int. Educ. 10, 241–266. doi: 10.1177/1028315306287002

Crossref Full Text | Google Scholar

Elshami, W., Taha, M. H., Abdalla, M. E., Abuzaid, M., Saravanan, C., and Al Kawas, S. (2022). Factors that affect student engagement in online learning in health professions education. Nurse Educ. Today 110:105261. doi: 10.1016/j.nedt.2021.105261

PubMed Abstract | Crossref Full Text | Google Scholar

Escobar-Pérez, J., and Cuervo-Martínez, Á. (2008). Validez de contenido y juicio de expertos: una aproximación a su utilización. Av. Medición 6, 27–36.

Google Scholar

García-Valdés, M., and Suárez-Marín, M. (2013). El método Delphi para la consulta a expertos en la investigación científica. Rev. Cub. Salud. Publica 39, 253–267. Available at: http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S0864-34662013000200007&lng=es&tlng=es

Google Scholar

George, D., and Mallery, P. (2001). SPSS for Windows: Stepby Step, 3rd. Edn. Boston. MA: Allyn and Bacon.

Google Scholar

Goto, K., and Gutierrez-Gomez, Y. (2024). Research globally, learn globally: an innovative course-based research project through collaborative online international learning (COIL) among nutrition students from American and Mexican Universities. Pedagogy Health Promot. 10, 56–62. doi: 10.1177/23733799231214960

Crossref Full Text | Google Scholar

Greller, W., Santally, M. I., Boojhawon, R., and Rajabalee, Sungkur (2017). Using learning analytics to investigate student performance in blended learning courses. J. High. Educ. Dev. 12, 37–63. doi: 10.3217/zfhe-12-01/03

Crossref Full Text | Google Scholar

Hackett, S., Janssen, J., Beach, P., Perreault, M., Beelen, J., van Tartwijk, J., et al. (2023). The effectiveness of collaborative online international learning (COIL) on intercultural competence development in higher education. Int. J. Educ. Technol. High. Educ. 20:5. doi: 10.1186/s41239-022-00373-3

PubMed Abstract | Crossref Full Text | Google Scholar

Hajimiri, K., Moradi, M., Hosseini, E., Jafari, M., and Nasehi, L. (2023). A conceptual model for empowering faculty members of third-generation universities of medical sciences. J. Taibah Univ. Med. Sci. 18:371. doi: 10.1016/j.jtumed.2022.10.002

PubMed Abstract | Crossref Full Text | Google Scholar

Helena, L., and Alena, P. (2021). Internationalisation as a strategy of educating future teachers of English. Philol. Class 26, 166–177.doi: 10.51762/1FK-2021-26-03-14

Crossref Full Text | Google Scholar

Hofhuis, J., Jongerling, J., Van der Zee, K. I., and Jansz, J. (2020). Validation of the Multicultural Personality Questionnaire Short Form (MPQ-SF) for use in the context of international education. PLoS ONE 15:e0244425. doi: 10.1371/journal.pone.0244425

PubMed Abstract | Crossref Full Text | Google Scholar

House, S. K., Nielsen, K., and Dowell, S. (2022). International collaboration using collaborative online international learning (COIL)/globally networked learning (GNL) model. Teach. Learn. Nurs. 17, 421–424. doi: 10.1016/j.teln.2022.06.010

Crossref Full Text | Google Scholar

Imran, R., Fatima, A., Salem, I. E., and Allil, K. (2023). Teaching and learning delivery modes in higher education: looking back to move forward post-COVID-19 era. Int. J. Manage. Educ. 21:100805. doi: 10.1016/j.ijme.2023.100805

Crossref Full Text | Google Scholar

Ingram, L. A., Monroe, C., Wright, H., Burrell, A., Jenks, R., Cheung, S., et al. (2021). Fostering distance education: lessons from a united states-england partnered collaborative online international learning approach. Front. Educ. 6:782674. doi: 10.3389/feduc.2021.782674

Crossref Full Text | Google Scholar

Liu, Y., and Shirley, T. (2021). Without crossing a border: Exploring the impact of shifting study abroad online on students' learning and intercultural competence development during the COVID-19 pandemic. Online Learn. J. 25:182. doi: 10.24059/olj.v25i1.2471

PubMed Abstract | Crossref Full Text | Google Scholar

López-Gómez, E. (2018). El método Delphi en la investigación actual en educación: una revisión teórica y metodológica. Educación XX1 21, 17–40. doi: 10.5944/educxx1.20169

Crossref Full Text | Google Scholar

Marcillo-Gómez, M., and Desilus, B. (2016). Collaborative online international learning experience in practice opportunities and challenges. J. Technol. Manage. Innov. 11, 30–35. doi: 10.4067/S0718-27242016000100005

PubMed Abstract | Crossref Full Text | Google Scholar

Mestre-Segarra, M. Á., and Ruiz-Garrido, M. F. (2022). Examining students' reflections on a collaborative online international learning project in an ICLHE context. System 105:102714. doi: 10.1016/j.system.2021.102714

Crossref Full Text | Google Scholar

Mok, K. H., and Marginson, S. (2021). Massification, diversification and internationalisation of higher education in China: critical reflections of developments in the last two decades. Int. J. Educ. Dev. 84:102405. doi: 10.1016/j.ijedudev.2021.102405

Crossref Full Text | Google Scholar

Mukul, E., and Büyüközkan, G. (2023). Digital transformation in education: a systematic review of education 4.0. Technol. Forecast. Soc. Change 194:122664. doi: 10.1016/j.techfore.2023.122664

Crossref Full Text | Google Scholar

Nasa, P., Jain, R., and Juneja, D. (2021). Metodología Delphi en la investigación sanitaria: how to decide its appropriateness. World J. Methodol. 11, 116–129. doi: 10.5662/wjm.v11.i4.116

PubMed Abstract | Crossref Full Text | Google Scholar

Nethsinghe, R., Joseph, D., Mellizo, J., and Cabedo-Mas, A. (2023). Teaching songs from diverse cultures to pre-service teachers using a “Four Step Flipped” method. Int. J. Music Educ. 41, 383–397. doi: 10.1177/02557614221110952

Crossref Full Text | Google Scholar

Noam, E. M. (1995). Electronics and the dim future of the university. Science 270, 247–249. doi: 10.1126/science.270.5234.247

Crossref Full Text | Google Scholar

Palacios-Núñez, M. L., Toribio-López, A., Robles-Mori, H., and Deroncele-Acosta, A. (2023). “Validity and reliability of the ACL21 for evaluating online collaborative learning,” in 2023 IEEE 3rd International Conference on Advanced Learning Technologies on Education and Research (ICALTER) (Chiclayo: IEEE), 1–4. doi: 10.1109/ICALTER61411.2023.10372932

Crossref Full Text | Google Scholar

Pouromid, S. (2019). Shaping learner responses in question-answer sequences in the EFL classroom. Int. J. Learn. Teach. Educ. Res. 18, 116–135. doi: 10.26803/ijlter.18.12.8

Crossref Full Text | Google Scholar

Rajabalee, B. Y., Santally, M. I., and Rennie, F. (2020). A study of the relationship between students' engagement and their academic performances in an eLearning environment. E-Learn. Digit. Media 17, 1–20. doi: 10.1177/2042753019882567

Crossref Full Text | Google Scholar

Razali, S. N., Shahbodin, F., Ahmad, M. H., and Nor, H. (2016). Measuring validity and reliability of perception of online collaborative learning questionnaire using rasch model. Int. J. Adv. Sci. Eng. Inform. Technol. 6, 966–974. doi: 10.18517/ijaseit.6.6.1343

Crossref Full Text | Google Scholar

Rubin, J., and Guth, S. (Eds.). (2022). The Guide to COIL Virtual Exchange: Implementing, Growing, and Sustaining Collaborative Online International Learning, 1st Edn. London: Routledge. doi: 10.4324/9781003447832-5

Crossref Full Text | Google Scholar

Salmon, J., Satoglu, E. B., Ogutu, V., and Thurston, P. (2022). Teaching international business skills across US and Kenya: a model for international collaboration. J. Teach. Int. Bus. 33, 127–148. doi: 10.1080/08975930.2022.2123427

Crossref Full Text | Google Scholar

Shimizu, I., Kikukawa, M., Tada, T., Kimura, T., Duvivier, R., Van Der Vleuten, C., et al. (2020). Measuring social interdependence in collaborative learning: instrument development and validation. BMC Med. Educ. 20, 1–9. doi: 10.1186/s12909-020-02088-3

PubMed Abstract | Crossref Full Text | Google Scholar

Sinclair, P. M., Levett-Jones, T., Morris, A., Carter, B., Bennett, P. N., and Kable, A. (2017). High engagement, high quality: a guiding framework for developing empirically informed asynchronous e-learning programs for health professional educators. Nurs. Health Sci. 19, 126–137. doi: 10.1111/nhs.12322

PubMed Abstract | Crossref Full Text | Google Scholar

Suarez, E. D., and Michalska Haduch, A. (2020). Teaching business with internationally built teams. J. Teach. Int. Bus. 31, 312–336. doi: 10.1080/08975930.2020.1851625

Crossref Full Text | Google Scholar

SUNY COIL CENTER (2013). Faculty Guide for Collaborative Online International. New York, NY: State University of New York.

Google Scholar

Tajik, F., and Vahedi, M. (2021). Quarantine and education: an assessment of iranian formal education during the COVID-19 outbreak and school closures. Int. J. Educ. Dev. Using Inf. Commun. Technol., 17, 159–175.

PubMed Abstract | Google Scholar

Tajpour, M., and Hosseini, E. (2021). Towards a creative-oriented university. J. Bus. Strategy Financ. Manag. 2:2. doi: 10.12944/JBSFM.02.01-02.03

Crossref Full Text | Google Scholar

Vahed, A., and Rodriguez, K. (2020). Enriching students' engaged learning experiences through the collaborative online international learning project. Innov. Educ. Teach. Int. 58, 596–605. doi: 10.1080/14703297.2020.1792331

Crossref Full Text | Google Scholar

Ventura-León, J. (2019). De regreso a la validez basada en el contenido. Adicciones 34, 323–326. doi: 10.20882/adicciones.1213

PubMed Abstract | Crossref Full Text | Google Scholar

Watla-iad, K., and Kradtap Hartwell, S. (2022). Creating a globally enhanced chemistry course through an intercultural collaborative assignment. J. Chem. Educ. 99, 1068–1075. doi: 10.1021/acs.jchemed.1c00696

Crossref Full Text | Google Scholar

Weller, L., Pfister, R., and Kunde, W. (2020). Anticipation in sociomotor actions: similar effects for in- and outgroup interactions. Acta Psychol. 207:103087. doi: 10.1016/j.actpsy.2020.103087

PubMed Abstract | Crossref Full Text | Google Scholar

West, H., Goto, K., Borja, S. A. N., Trechter, S., and Klobodu, S. (2022). Evaluation of a Collaborative Online International Learning (COIL): a food product analysis and development project. Food Cult. Soc. 27, 152–173. doi: 10.1080/15528014.2022.2069441

Crossref Full Text | Google Scholar

Wilcox, R., and Serang, S. (2017). Hypothesis testing, p values, confidence intervals, measures of effect size, and Bayesian methods in light of modern, robust techniques. Educ. Psychol. Meas. 77, 673–689. doi: 10.1177/0013164416667983

PubMed Abstract | Crossref Full Text | Google Scholar

Wimpenny, K., and Orsini-Jones, M. (2020). “Innovation in collaborative online international learning: a holistic blend,” in Radical Solutions and eLearning: Practical Innovations and Online Educational Technology, ed. D. Burgos (Cham: Springer), 1–25. doi: 10.1007/978-981-15-4952-6_1

Crossref Full Text | Google Scholar

World bank World Development Indicators. (2024). Datos. Inscripción escolar, nivel terciario (%bruto). Paris: Instituto de Estadística de la Organización de las Naciones Unidas para la Educación, la Ciencia y la Cultura (UNESCO). Available at: https://datos.bancomundial.org/indicador/SE.TER.ENRR

Google Scholar

Zapatero, A. S.M., Valle, C. D. G., and León, O. M. (2022). Aprendizaje colaborativo internacional en línea (COIL) en la formación inicial del profesorado en didáctica de las ciencias sociales. Rev. Educ. Distanc. 22, 1–18. doi: 10.6018/red.521651

Crossref Full Text | Google Scholar