Using technology-based vocabulary instruction inside and outside of the classroom

Introduction: The current pre-test post-test study explored the use of WhatsApp for vocabulary acquisition inside and outside the English as a Foreign Language classroom.

Methods: Native Arabic undergraduate students learned English vocabulary items both inside and outside of the classroom using WhatsApp groups. Pre- and post-tests gauged participants, knowledge of the vocabulary items used in the intervention as well as control items. An additional questionnaire explored participants' satisfaction with using WhatsApp inside and outside of the classroom.

Results: We found a small, but significant, vocabulary learning effect for using WhatsApp outside of the classroom, but no such effect for the use of WhatsApp inside the classroom. The questionnaire results additionally showed that students were highly satisfied with the use of WhatsApp to learn vocabulary, especially in terms of instructor availability. Finally, we found that students who were more motivated to use WhatsApp inside the classroom showed higher learning gains inside the classroom than those less motivated.

Discussion: We discuss the results with respect to the role of space and motivation for vocabulary acquisition using technologies such as WhatsApp.

Introduction

Mobile devices are ubiquitous in modern everyday life. For example, an average person spends around 3.2 h per day on their cell phone and 90% of this time using applications (Blair, 2020). A majority of cell phone users are young people between the ages of 18 and 29 (Marko, 2023). This regular use of mobile devices among young users has led teachers, researchers and educators to note the potential to use mobile applications in educational contexts, for example to support the learning of English as a Foreign Language (EFL) and to develop learners' English language skills (Taki and Khazaei, 2011; Hsu et al., 2013). Mobile devices are not only ubiquitous, but also portable, multimodal, and interactive and can thus be used to allow students to learn autonomously in a multimodal and interactive fashion (Klimova, 2021).

One particular area of difficulty for EFL learners is vocabulary acquisition, such that many EFL learners have limited vocabulary knowledge (Alfadil, 2020). Various studies suggest that the breadth of EFL learners' vocabulary knowledge, i.e., the number of words that learners know, relates closely to their language proficiency (Nation, 2001; Shen, 2008; Koizumi and In'nami, 2013). As such, recent research has explored how vocabulary learning strategies can be used to expand learners' lexical knowledge (e.g., Hamzah et al., 2009; AlSaif, 2011; Alahmadi et al., 2018; Alahmadi and Foltz, 2020). Other recent research has explored how mobile applications, particularly instant messaging applications, can be used to facilitate learners' vocabulary acquisition (Song and Chen, 2017; Mahdi, 2018). A particularly popular instant messaging application is WhatsApp (https://www.whatsapp.com/), which is currently used in more than 180 countries by over two billion people, who send approximately 100 billion messages daily (Ceci, 2022a). In Saudi Arabia, where we conducted the current study, almost 90% of internet users use WhatsApp (Ceci, 2022b). WhatsApp is also one of the most common applications used by college students (Jadhav et al., 2013), the participants in our study. WhatsApp allows people to send text, picture, video and audio messages and might be a well-suited and engaging application for educational endeavors (So, 2016). In the current study, we therefore explore how WhatsApp can be used inside and outside of the classroom for EFL learners' vocabulary acquisition.

Using WhatsApp for vocabulary acquisition inside the classroom

In this section, we focus on how WhatsApp can be used for vocabulary acquisition inside the classroom. In an EFL context, social media and other recent technologies, such as WhatsApp, can enrich group interaction, facilitate discussions, and improve cooperation between learners (Bryer and Zavattaro, 2011). For instance, EFL instructors can use instant messaging to provide students with a learning community, for example by creating groups, and to foster positive attitudes toward classroom activities (Rau et al., 2008). Instant messaging also allows teachers to provide learning material in small chunks and to introduce target information in a potentially less overwhelming manner. Furthermore, a sense of privacy when using WhatsApp (Church and De Oliveira, 2013) might encourage those students to participate who might otherwise be too shy to speak up in class.

Interestingly, very few studies have looked at how WhatsApp could be used successfully in the classroom to learn vocabulary. Dehghan et al. (2017) compared the use of WhatsApp to learn vocabulary in the classroom with traditional classroom instruction. The participants were 32 EFL students aged between 13 and 16 years of age. One group of students learned textbook vocabulary using traditional classroom instruction, while the other group learned the same textbook vocabulary items using WhatsApp complemented with classroom discussion. Participants in both groups performed similarly in a multiple-choice post-test, suggesting that the use of WhatsApp in the classroom was not any more effective than traditional classroom instruction. Jafari and Chalak (2016) explored whether WhatsApp could support lexical acquisition in the junior high school classroom. A total of 60 Iranian EFL students learned vocabulary items in the classroom either through a WhatsApp chat group with multimodal teaching materials or using traditional textbook-based in-class vocabulary teaching methods. Both groups of students performed similarly in a multiple-choice vocabulary pre-test, but the WhatsApp group significantly outperformed the traditional group in the post-test. This suggests that, in contrast to Dehghan et al.'s (2017) results, WhatsApp can be an efficient tool to expand learners' vocabulary knowledge. Overall, very few studies have looked at using WhatsApp in the classroom to learn vocabulary, and those studies that have been conducted have yielded conflicting results. How useful WhatsApp may be for vocabulary acquisition in the classroom is thus an open question.

Using WhatsApp for vocabulary acquisition outside the class

We now move on to how WhatsApp can be used for vocabulary acquisition outside the classroom. WhatsApp allows learners to easily access obtained contents at a time that is convenient for them. Furthermore, users frequently check their phones upon hearing the sound that alerts them that they have a message, which teachers might use to attract learner's attention to the target materials (Jones et al., 2009). Conversely, WhatsApp also allows for prompt responses by instructors, which have been found to increase learners' motivation levels and cognitive learning (Allen et al., 2006). Finally, learners have reported that smartphone applications, such as WhatsApp, help them to learn vocabulary informally, i.e., outside of an educational setting (Al-Sofi, 2020). WhatsApp might therefore also be an effective tool to learn vocabulary items outside of the classroom in a more formal manner.

Previous studies do indeed support the usefulness of WhatsApp for vocabulary learning outside of the classroom. For example, Fageeh (2013) explored the use of WhatsApp to complete vocabulary homework assignments (see Bensalem, 2018, for a similar study with similar results). A total of 58 Saudi undergraduate students participated in the study. One group received a list of target words via WhatsApp and was asked to look up the words using an online dictionary and employ them in sentences that they would post in the WhatsApp group for peer and instructor feedback. The other group received the same list of words in class, completed the same activity (looking up words and writing sentences with these words) at home and then turned in their sentences for instructor feedback. While both groups performed similarly in a pre-test, the WhatsApp homework group significantly outperformed the traditional homework group in a post-test. These results suggest that WhatsApp can be used successfully for homework assignments that practice vocabulary outside of the classroom.

Ashiyan and Salehi (2016) explored the use of WhatsApp to learn English collocations outside of the classroom. Sixty Iranian EFL learners at the intermediate level were divided into a WhatsApp and a control group. Both groups were taught collocations in the classroom using a regular textbook. The WhatsApp group additionally used WhatsApp to learn collocations outside of the classroom with native and non-native speakers of English around the world. Both groups of learners performed similarly in a pre-test gauging their knowledge of English collocations, but the WhatsApp group significantly outperformed the control group in the collocation post-test. This suggests that WhatsApp can be used successfully outside of the classroom to improve learners' collocation knowledge. However, since the control group received no additional targeted practice outside of the classroom, it is not clear whether Ashiyan and Salehi's (2016) results are due to the efficacy of WhatsApp for learning collocations or simply due to students in the WhatsApp group having had more opportunities overall to practice collocations.

Basal et al. (2016) investigated how WhatsApp compares with traditional classroom activities when it comes to learning idioms. Fifty Turkish first-year university students participated in the study. The WhatsApp group received multimedia messages to practice the idioms (explanation of meaning, picture demonstrating the idiom, example sentences, fill in the blank exercises etc.) outside of the classroom. The control group received printed handouts with traditional exercises practicing idioms (explanation of meaning, example sentences, fill in the blank exercises etc.) inside the classroom. Both groups performed similarly on a pre-test, but the WhatsApp group outperformed the control group in a post-test, suggesting that WhatsApp can be used successfully to learn idioms. However, as with the previous study, it is possible that the WhatsApp group simply had more learning opportunities than the control group, as they received additional exercises outside of the classroom. It is thus again not clear if the higher learning effect in the WhatsApp group relates to the number of learning opportunities or the effectiveness of WhatsApp.

Çetinkaya and Sütçü (2018) compared the effectiveness of providing additional vocabulary information via WhatsApp and Facebook compared to regular face-to-face classes only. A total of 123 Turkish high school students participated. Participants in the WhatsApp and Facebook groups received messages or posts, each with an English vocabulary item, an English definition of the word, a Turkish translation of the word, and an English example sentence using the word. The messages and posts, respectively, were sent between 8:00 a.m. and 9:30 p.m., suggesting that participants received at least some of the messages outside of the classroom. The control group received no intervention. The results showed larger learning effects from pre-test to post-test for the WhatsApp group compared to both the Facebook group and the control group, suggesting that additional practice using WhatsApp was more successful than additional practice using Facebook.

Finally, Rashtchi and Yazdani (2020) examined vocabulary acquisition through WhatsApp voice messages vs. written messages received outside of the classroom. Participants were 50 female EFL students aged between 16 and 17 years. Both groups learned novel vocabulary items through messages sent to a WhatsApp group. For the voice message group, the teacher audio recorded messages in which she pronounced the new words and provided definitions, synonyms, and antonyms. For the text message group, the teacher wrote messages about the new vocabulary items. She highlighted the new words and provided definitions, synonyms, and antonyms. The results of the pre-test and post-test showed significant learning effects of similar magnitude in both groups. Thus, there was no evidence that the type of message (voice vs. text) affected learning outcomes. Therefore, the researchers concluded that WhatsApp is an effective tool for learning vocabulary regardless of the input modality. Overall, the previous studies provide evidence that WhatsApp can be used successfully to learn vocabulary outside of the classroom.

Advantages and disadvantages of using WhatsApp for vocabulary acquisition

Using WhatsApp for vocabulary acquisition can have both advantages and disadvantages. For instance, digital discussions via instant messaging could promote brainstorming and accelerate mutual understanding (Hwang et al., 2011). Instant messaging can also increase active learning, offer immediate feedback, reduce time spent on tasks, and provide authorized access to academic materials (Farmer, 2003; Desai and Graves, 2006). The use of mobile applications can also combine formal and informal education approaches and provide students with multidimensional learning activities (Cook et al., 2009). Furthermore, employing WhatsApp in educational settings could give less confident or shy learners an opportunity to engage with the learning materials as well as with other learners in a non-threatening way (Rambe and Bere, 2013). Thus, WhatsApp might increase learners' motivation levels and encourage them to participate in learning activities (Fageeh, 2013). WhatsApp can further be beneficial for busy or independent learners as it allows them to access needed materials at any time (So, 2016). Overall, WhatsApp can provide learners with an interactive and flexible learning experience that can cater to individual students' needs, and students tend to have positive views regarding the use of WhatsApp to learn vocabulary (Fageeh, 2013; Çetinkaya and Sütçü, 2018).

Although there are advantages of using instant messaging for educational purposes, there are some drawbacks. For instance, instant messaging is best suited for short messages, which might limit its learning effect as it might be difficult to convey or understand more complex information at such short length (Hill et al., 2007). Another issue is learners' attention while studying via an instant messaging application such as WhatsApp. For example, students might receive other messages, either in WhatsApp or through other applications, while they are studying and these messages can interrupt their learning and draw their attention away from the task at hand (Markett et al., 2006). Moreover, there are concerns regarding learners' personal space and inappropriate timing for messaging (cf. Çetinkaya and Sütçü, 2018), as some learners thought that the use of instant messaging applications in education encroached on their personal space (Brett, 2011). There may also be some technical issues, such as internet speed and/or short battery life (So, 2016).

The current study

This study is designed to examine the effectiveness of WhatsApp as a tool for improving vocabulary knowledge of EFL learners in Saudi Arabia. This is particularly important given the dominant role of explicit grammar teaching and intentional lexical acquisition in the EFL context of Saudi Arabia. In this context, learners are primarily focused on memorizing target rules or lexical items, with limited attention paid to individual differences or abilities. As a result, there is a need to explore alternative approaches that may better support learner outcomes.

The study will compare the effectiveness and perceived value of WhatsApp usage in two contexts: inside and outside the classroom. The comparison is motivated by the desire to understand how the use of social network applications can contribute to vocabulary acquisition and retention in the EFL context. Given the widespread use of instant messaging apps like WhatsApp, it is important to explore their potential impact on EFL learning. Previous studies suggest that WhatsApp can benefit vocabulary acquisition outside of the classroom, but there is less evidence for its effectiveness inside the classroom. However, no previous study has directly compared the use of WhatsApp for vocabulary acquisition inside and outside of the classroom. Furthermore, based on the results of previous studies, it is not always clear if the learning effects are due to additional practice in general or the use of WhatsApp in particular. The current study aims to fill these gaps. We also hope that the results of our study will shed light on the role of WhatsApp in building EFL learners' vocabulary knowledge and inform the ongoing development of effective EFL pedagogy in Saudi Arabia.

In the current study, we ask the following research questions:

RQ1: Do students learn vocabulary items successfully through WhatsApp when used in the classroom and when used outside of the classroom?

RQ2: How do students rate the experience of learning vocabulary through WhatsApp?

RQ3: Do students who report a more positive learning experience using WhatsApp show larger vocabulary learning effects?

Based on the previous literature, we expect that using WhatsApp outside the classroom may result in a greater learning effect than using WhatsApp inside the classroom, possibly because it allows learners to engage with the language in a more authentic and unstructured setting. We also expect that students will overall have a positive experience learning vocabulary through WhatsApp due to its convenience and accessibility as a mobile app. Additionally, students may rate the use of WhatsApp inside the classroom more favorably, as it may have a more direct connection to their language learning goals. We further hypothesize that students who have a more positive experience using WhatsApp will exhibit a larger vocabulary learning effect, as a positive attitude toward the learning method is likely to lead to increased motivation and engagement.

Methods

Participants

A total of 44 Saudi adult EFL undergraduate students participated in the study. They were recruited from two second-year English major classes at the English Department of King Abdulaziz University. Half of the participants were 21 or 22 years of age, with another nine participants aged between 24 and 26, eight aged between 18 and 20 and the remaining five participants aged over 26. On average, participants self-rated their English proficiency as intermediate, with an average rating of 2.11 (SD = 0.72) on a scale of 1 = beginner, 2 = intermediate, 3 = advanced, 4 = fluent, and 5 = near native. In fact, 35 of the 44 participants rated their English proficiency as either intermediate or advanced. On average, participants reported sometimes using English outside the classroom, with an average rating of 2.63 (SD = 1.01) on a scale of 1 = never, 2 = rarely, 3 = sometimes, 4 = frequently, and 5 = always. Almost half of the 44 participants (N = 21) reported sometimes using English outside the classroom. Using the same rating scale (1 = never to 5 = always), participants also provided information about how confidently they can engage in certain tasks in English. On average, participants reported that they can sometimes freely write their opinion on a variety of topics, with an average rating of 2.70 (SD = 1.09). Sometimes was also the most frequent response, given by exactly 50% (N = 22) of the participants. They also reported that they can sometimes take notes during lectures, again with an average rating of 2.70 (SD = 1.00). Here, the most common responses were sometimes (N = 18) and frequently (N = 11). Participants are slightly more confident in their essay writing abilities, with most participants reporting that they can sometimes (N = 17) or frequently (N = 14) build up their argument in a logical way within an essay, yielding an average rating of 3.43 (SD = 1.07).

Materials and procedure

The materials for the study involved a short online English proficiency test (https://www.efset.org/quick-check/) to ensure that participants' English was at an appropriate level, a vocabulary pre-test, WhatsApp training tasks, a vocabulary post-test, and a questionnaire collecting demographic information and gauging participants' opinions about using WhatsApp for vocabulary acquisition.

The vocabulary pre- and post-tests included a list of English vocabulary items. Participants were asked to write the Arabic translations for all the words that they knew and to cross out the words that they didn't know. Participants were not allowed to use a dictionary for this task. Both the pre- and the post-test included 62 words of which 30 were target items, which participants would practice using WhatsApp either in the classroom or outside of the classroom. The 30 target items were distributed across two sets of 15 words, such that half the participants would train the words in Set 1 outside of the classroom and the other half inside the classroom and vice versa for Set 2. The pre- and post-tests additionally included 10 control items (pre-post control), for which participants would receive no further practice. The pre-test additionally included 22 filler items. In contrast, the post-test included an additional 10 control items (post control) that only occurred in the post-test as well as 12 filler items. Due to an error, the tests included some unintended words, such that the target and control items are not matched as closely as intended. Specifically, the average frequencies in the British National Corpus (https://www.english-corpora.org/bnc/) was 175 (SD = 96) for the post control words, 312 (SD = 221) for the pre-post control words, 176 (SD = 230) for the words in training Set 1, and 176 (SD = 209) for the words in training Set 2. Despite not being matched as well as intended, the results from a generalized linear model (family = quasipoisson, for overdispersed count data) showed that the average frequencies across the four conditions did not differ significantly (β = −0.08, SE = 0.14, t = −0.58, p = 0.562). The words across the conditions were also well matched in length, with an average of 7 letters for the post control and pre-post control words and an average of 8 letters for the words in training Sets 1 and 2. A generalized linear model (family = poisson, for count data) showed that the average word length across the four conditions did not differ significantly (β = 0.04, SE = 0.05, t = 0.76, p = 0.450). Vocabulary items included nouns, verbs and adjectives, and these parts of speech were also distributed similarly across the four conditions. Specifically, the post control words included 4 nouns, 3 verbs, and 3 adjectives, the pre-post control words involved 2 nouns, 4 verbs, and 4 adjectives, and both training sets included 5 nouns, 6 verbs, and 4 adjectives. A Chi-square test suggests that the four conditions have a similar distribution of these three parts of speech (χ = 1.32, p = 0.971). Overall, the words in the four conditions are thus rather well matched.

All participants completed two WhatsApp training tasks. Half the participants learned the target words in Set 1 in a WhatsApp group in the classroom and the target words in Set 2 in the WhatsApp group outside of the classroom, whereas the other half of the participants learned the target words in Set 2 in a WhatsApp group in the classroom and the target words in Set 2 in the WhatsApp group outside of the classroom. Participants completed these tasks during 1 week in which they received words to learn inside of the classroom and words to learn outside of the classroom each day. Participants received a total of 50 words to learn via WhatsApp, 25 inside the classroom and 25 outside of the classroom. Of these, 30 words were target words (15 inside the class and 15 outside) and the remaining 20 words were filler words. The teacher asked the students to write a meaningful sentence with each of the words that they received that day. The teacher then provided feedback on grammar and vocabulary. In the classroom, students had 60 min to finish the activity. Outside of the classroom, students were told to not spend more than 60 min on the task, but they were flexible in that they could complete the activity at any time before 4 p.m. that day. We provided this flexibility as it is not easy to ask students to participate in a WhatsApp group outside of class during one specific 60-min period. We set the 4 p.m. deadline so that teachers could give immediate feedback to students at starting at 4 p.m. every day.

The questionnaire included questions about participants' language background. The questionnaire also explored students' attitudes toward studying vocabulary using an instant messaging application to acquire new vocabulary inside or outside the classroom. To gauge this, participants provided their agreement with 10 statements about using the WhatsApp group inside the classroom and 10 analogous statements about using the WhatsApp group outside the classroom (see Table 3 below). The response options were 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree.

Participants first completed the brief English proficiency test and the vocabulary pre-test. In the following week, participants learned the target vocabulary items via WhatsApp. About a week later, participants completed the vocabulary post-test and filled in the questionnaire. The anonymized data as well as the analysis scripts are available on the Open Science Framework at: https://osf.io/4k2tn/.

Results

Vocabulary learning

We first analyze whether students learned vocabulary successfully through WhatsApp when used in the classroom and when used outside of the classroom (RQ1). We begin with the results from the post-test to see if students knew more words that were trained using WhatsApp than control words in the post-test. Figure 1 shows the proportion of correct responses for the post-test for the control words, which participants did not learn via WhatsApp, and for the target words, which participants learned via WhatsApp either inside or outside of the classroom. The figure shows that participants knew 39.5% of the control words that only appeared in the post-test (post control condition) and 41.8% of words that appeared in the pre- and post-tests (pre-post control condition). The figure also shows that participants knew 52.1% of the words that they learned in the classroom and 53.8% of the words that they learned outside of the classroom.

FIGURE 1

Figure 1. Proportion of correct responses for the control and learning conditions for the post-test.

To see if the proportion of words known in the post-test differed across conditions, we ran a mixed logit model (family = binomial) with the number of successes and failures (i.e., correct and incorrect responses) as response variable and condition as fixed factor. For all of our analyses, we used sum-coding for all fixed factors for ANOVA-style main effects and interactions. We also added participant to the model as a random effect. The results show a significant main effect of condition (β = 0.32, SE = 0.052, z = 6.29, p < 0.001). To see which of the four conditions differ significantly from each other, we ran pairwise comparisons, using the emmeans function in R. Table 1 shows the results from the pairwise comparisons and suggests that participants had significantly more correct responses for the two learning conditions compared to the two control conditions in the post-test. However, post-test performance was similar when comparing the two control conditions and when comparing the two learning conditions. This suggests that participant had significantly better post-test knowledge of the words that they learned using WhatsApp compared to the words that they did not learn using WhatsApp. Similar amounts on learning occurred for using WhatsApp inside and outside of the classroom.

TABLE 1

Table 1. Results from the pairwise comparisons for the post-test.

Since participants knew some of the target and control words already during the pre-test, we now consider the amount of learning that occurred from pre-test to post-test for all the words that occurred both in the pre-test and the post-test, i.e., for the pre-post control condition as well as the two learning conditions. Figure 2 shows the proportion of words that participants learned from pre-test to post-test for the control condition and the two learning conditions. The figure shows that the words that participants knew increased by 39.1% for the pre-post control condition, 45.2% for the classroom learning condition, and 48.0% for the outside of the classroom learning condition.

FIGURE 2

Figure 2. Proportion of words learned from pre-test to post-test for the control and learning conditions.

To see if the proportion of words learned from pre-test to post-test differed across conditions, we ran a linear mixed model with the proportional learning effect as response variable. We calculated the proportional learning effect by subtracting the number of correct responses in the pre-test from the number of correct responses in the post-test and dividing the result by the number of possible correct responses. For example, if a participant had 2 correct responses in the classroom condition in the pre-test and 4 correct responses in the classroom condition in the post-test, then the proportional learning effect was (4 – 2)/15 = 0.13. We chose a linear model rather than a logit model since the proportional learning effect can yield negative values. The model also had condition as a fixed factor and participant as a random factor. The result shows a significant main effect of condition (β = 0.04, SE = 0.01, z = 2.50, p = 0.014). To see which of the three conditions differ significantly from each other, we ran pairwise comparisons, using the emmeans function in R. Table 2 shows the results from the pairwise comparisons and suggests that participants' learning effect was significantly larger for the outside of the classroom learning condition compared to the pre-post control condition. However, the amount of learning was similar for the pre-post control condition and the classroom learning condition as well as for the classroom learning and outside learning conditions. This suggests that only the use of WhatsApp outside of the classroom yielded a significant learning effect compared to the control condition. While using WhatsApp inside the classroom led to numerically more learning than the control condition, this increase was not statistically significant.

TABLE 2

Table 2. Results from the pairwise comparisons for the learning effect.

Student satisfaction

We now turn to how students rated the experience of learning vocabulary through WhatsApp both inside and outside of the classroom (RQ2). Table 3 shows participants' average agreement with a set of statements about using the WhatsApp group inside the classroom and outside of class. Responses were on a Likert scale from 1 = strongly disagree to 5 = strongly agree. The table shows that participants showed general agreement with all statements, suggesting that they were highly satisfied with the use of WhatsApp both inside and outside of the classroom to learn vocabulary items. We ran dependent-samples sign tests to see if participants' agreement with any of the statement was higher when it concerned using WhatsApp in the classroom compared to outside of the classroom. Table 3 reveals no significant p-values, suggesting that student satisfaction was equally high for the use of WhatsApp inside the classroom and outside the classroom. The results for the last statement (Using the WhatsApp group inside the class allows me to interact better than in the WhatsApp group outside the class and vice versa), however, suggest that participants' ratings need to be treated with some caution. It seems that participants should have agreed with one of the statements, but not the other (or remained neutral for both). Instead, participants overall agreed with both statements. A Spearman correlation test also showed a significant positive correlation (S = 9,183.3, p = 0.019) for ratings of the two versions of the statement, suggesting that participants who agreed with one of the statements tended to also agree with the other. This suggests that participants may have agreed with most of the statements to please their instructors, who were running the WhatsApp group and also administering the questionnaire.

TABLE 3

Table 3. Participants' average agreement with statements about using the WhatsApp group inside the classroom and outside of class.

Factors influencing vocabulary learning

Finally, we explore if students who report a more positive learning experience using WhatsApp show larger vocabulary learning effects than students who report a less positive learning experience using WhatsApp (RQ3). We operationalize this research question by looking at whether participants who showed stronger agreement with the statements Using the WhatsApp group motivates me to learn English, Using the WhatsApp group helps me remember the new words, and I feel more motivated to complete my vocabulary homework using the WhatsApp group inside the class / at any time and any place also had larger vocabulary learning effects. We analyze this separately for the two learning scenarios, i.e., using WhatsApp inside the classroom and outside the classroom. We ran linear models with the proportional learning effect as response variable and agreement with the three statements as independent variables. Independent variables were centered to avoid multicollinearity. Again, we chose linear models because the proportional learning effect can yield negative values. Table 4 shows the results of the analyses and shows that, for the use of WhatsApp in the classroom, students who showed stronger agreement with the statement Using the WhatsApp group motivates me to learn English had significantly larger learning effects than students who showed less strong agreement with this statement. Agreement with the other two statements did not significantly affect the amount of learning. For use of WhatsApp outside of the classroom, there were no significant effects, such that we found no evidence of agreement to these statements influencing learning outcomes.

TABLE 4

Table 4. Results of the linear models for factors influencing vocabulary learning.

Discussion

In the current study, we explored how college students can learn vocabulary items using WhatsApp inside vs. outside of the classroom (RQ1). Our post-test results suggest that students know significantly more vocabulary items that they learned through WhatsApp both in the classroom and outside the classroom than control items. However, participants' pre- to post-test learning effect was only significantly larger than the control condition for words learned through WhatsApp outside of the classroom. This suggests that students can learn vocabulary items successfully when using WhatsApp outside of the classroom and, to some extent, when using WhatsApp inside the classroom. Students also rated their experience of learning vocabulary through WhatsApp (RQ2). We found that students were highly satisfied with the use of WhatsApp to learn vocabulary items, with no differences in satisfaction ratings for the use of WhatsApp inside vs. outside of the classroom. Finally, we explored whether students who reported a more positive learning experience using WhatsApp showed larger vocabulary learning effects (RQ3). Our analyses focused on students' motivation and how well they thought using WhatsApp helped them remember words. We found that students who agreed more strongly with the statement that using the WhatsApp group in the classroom motivated them to learn English had higher learning gains than students who agreed less strongly to this statement. Thus, higher motivation relates to a larger learning effect when WhatsApp is used in the classroom, but we found no such effect for using WhatsApp outside of the classroom. We will discuss these results in more detail in the following sections.

Vocabulary learning

We found a significant learning effect only for using WhatsApp outside of the classroom, but not inside of the classroom. This is largely in line with the previous literature, which found reliable learning effects for using WhatsApp outside of the classroom (e.g., Fageeh, 2013; Ashiyan and Salehi, 2016; Basal et al., 2016; Çetinkaya and Sütçü, 2018; Rashtchi and Yazdani, 2020), but conflicting results for using WhatsApp inside the classroom (Jafari and Chalak, 2016; Dehghan et al., 2017). Our study contributes novel data in that it directly compared the two learning situations. We also used a within-participant design, such that all participants engaged in using WhatsApp both inside and outside of the classroom. As a result, students in both groups had the same learning opportunities both inside and outside of the classroom; the groups merely differed in which words they learned inside vs. outside of the classroom. Thus, our study ruled out that the larger learning effect that we found for using WhatsApp outside compared to inside of the classroom was due simply to more learning opportunities, i.e., more practice or additional exercises outside of the classroom. Instead, participants did the exact same tasks inside and outside of the classroom. That said, participants were told to complete the tasks in the classroom in a certain time frame, and we could not reasonably enforce a 1-h time limit outside of the classroom. We therefore can't rule out that participants spend more than 1 h doing the tasks outside of the classroom and that this led to the significant learning effect.

We should also note that even though we found a significant learning effect for using WhatsApp outside of the classroom compared to control words, this learning effect seems to be rather small. Specifically, participants improved considerably from pre-test to post-test even for the control words, and the additional learning effect that we saw for using WhatsApp outside of the classroom is substantially smaller than this overall pre-test to post-test learning effect. There are several possible reasons for this. First, participants may have learned some vocabulary items incidentally over the course of the study. Participants may also have looked up and learned unknown words that occurred in the pre-test out of curiosity, which would improve post-test performance in general. Participants may also have provided more correct responses in general during the post-test compared to the pre-test because they were already familiar with the testing format. The instructions for the pre- and post-tests also asked participants to write the Arabic meaning for the words that they know and to cross out the words that they did not know. Participants may have followed these instructions more closely during the pre-test and engaged more frequently in guessing the Arabic meaning of words rather than crossing words out during the post-test.

Student satisfaction

Overall, we found that students were highly satisfied with the use of WhatsApp to learn vocabulary, regardless of whether WhatsApp was used inside or outside of the classroom. This is in line with many previous studies (e.g., Allen et al., 2006; Fageeh, 2013; Çetinkaya and Sütçü, 2018; Fahruddin et al., 2022) in which learners report positive attitudes toward the use of WhatsApp as a learning instrument. There are several possible reasons that might explain the overall positive attitude of our participants toward using WhatsApp both inside and outside of the classroom. First, 90% of internet users in Saudi Arabia use WhatsApp (Ceci, 2022b), such that our participants were most likely very familiar with the application and most likely used it on a daily basis. As a result, they likely felt confident to use WhatsApp for educational purposes. Notably, looking at the statements with the highest levels of agreement, we can see that participants especially appreciated that WhatsApp allowed them to contact their instructor and get immediate feedback from their instructor (cf., Allen et al., 2006). Participants also reported feeling more motivated to complete their vocabulary homework. This suggests that participants especially valued that the interactive and immediate nature of WhatsApp made interactions with their instructors easier, and this in turn may have motivated them to use WhatsApp for vocabulary work. The use of WhatsApp might have encouraged motivation in several ways, for example, by fostering curiosity, exploration and task completion (Yearta, 2012). Since students interacted with each other and with their instructor in a WhatsApp group, students may also have experienced high levels of collaborative learning, flexibility, and informal communication. This could additionally have contributed to the overall high satisfaction toward WhatsApp as a learning tool (Bouhnik and Deshen, 2014) that we found in the current study.

As already mentioned, we were surprised that participants agreed with both the statement that using the WhatsApp group inside the class allows them to interact better than in the WhatsApp group outside the class and the statements that using the WhatsApp group outside the class allows them to interact better than in the WhatsApp group inside the class. This seems at first contradictory. We already mentioned that participants may have agreed to both statements to please their instructor. However, since the questionnaire was conducted anonymously (with participants using participant codes rather than their names), the instructors would not know individual participants' responses, such that there was little incentive to try to please the instructors. Another possible explanation for this perplexing result is that participants were highly satisfied with using WhatsApp regardless of location and therefore agreed with both statements.

Overall, the use of an application like WhatsApp for educational determinations seems to be both beneficial educationally, since we found a significant learning effect for the use of WhatsApp outside of the classroom, and psychologically, since students were highly satisfied using WhatsApp to learn vocabulary. Thus, the use of WhatsApp can foster student engagement with classroom activities and make essential resources more accessible (Rau et al., 2008).

Factors influencing vocabulary learning

Lastly, we examined the impact of students' experiences with using WhatsApp on their vocabulary learning outcomes. We focused on how motivation and the perceived helpfulness of WhatsApp might influence students' learning outcomes. The results suggest that higher motivation can lead to larger vocabulary learning effects when WhatsApp is used in the classroom. However, we found no significant effect for statements for using WhatsApp outside of the classroom. These differences in results may relate to the physical differences across the two learning situations. Specifically, in the classroom students could alternatively interact with each other directly, i.e., face-to-face, rather than through WhatsApp. Some participants may have preferred to just talk to their instructor and classmates rather than use an application in a situation where everyone is physically co-present. These students might also have seen WhatsApp as a substitute for their instructor (cf., Stowe, 2013; Ashiyan and Salehi, 2016). These students may have both been less motivated to use WhatsApp in the classroom and might also have benefited more from personal face-to-face interaction in the classroom. In contrast, students who benefit less from face-to-face classroom interaction or who might be too shy to engage in the classroom face-to-face might have welcomed the use of WhatsApp in the classroom and were, in turn, very motivated to use WhatsApp in the classroom. Outside of the classroom presents a different situation because there is no face-to-face alternative for communication. Thus, in this context, WhatsApp creates the opportunity to interact with others rather than replaces a different means of interacting with each other. In the absence of a viable alternative, WhatsApp might improve learning outcomes regardless of motivation levels.

Overall, our findings suggest that students who have a positive learning experience using WhatsApp in the classroom show larger vocabulary learning effects. This highlights the potential of WhatsApp as a vocabulary acquisition tool in language learning in the classroom, but its use should be guided by clear educational goals. Specifically, applications should be used such that they do not replace the instructor (Stowe, 2013) and make good use of the limited time in which students are co-present in a physical space. Further research is needed to understand the factors that influence vocabulary learning outcomes using WhatsApp and other technology-enhanced learning tools, such as the quality of instruction, interaction with classmates and teachers, cultural background of students, and availability of technology.

Conclusions

In the current study, we looked at the use of WhatsApp for vocabulary acquisition inside and outside of the classroom. We found a small, but significant, learning effect for the use of WhatsApp outside of the classroom that was not affected by motivation. While the WhatsApp learning effect inside of the classroom did not differ significantly from a control, more motivated students did learn vocabulary more successfully in the classroom using WhatsApp than less motivated ones. Our results suggest an interplay between the physical space in which an application is used for learning and the motivation that learners bring to the task. Both seem to affect learning outcomes in different ways.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: Open Science Framework: https://osf.io/4k2tn/.

Ethics statement

The studies involving human participants were reviewed and approved by the English Department of King Abdulaziz University. The patients/participants provided their written informed consent to participate in this study.

Author contributions

AA and SA collected and coded the data and prepared the data for analysis. AF performed the statistical analyses. All authors contributed to the conception and design of the study, wrote sections of the first draft of the manuscript, contributed to manuscript revision, read, and approved the submitted version.

Funding

The Deanship of Scientific Research (DSR) at King Abdulaziz University (KAU), Jeddah, Saudi Arabia has funded this Project under grant no (G: 108-662-1443). The open access publication fees were covered through financial support of the University of Graz.

Acknowledgments

We would like to thank all the students who volunteered to participate in the study. We would also like to thank Mr. Monther Alluhaidah for his assistance during the data collection process.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Alahmadi, A., and Foltz, A. (2020). Effects of language skills and strategy use on vocabulary learning through lexical translation and inferencing. J. Psycholinguist. Res. 49, 975–991. doi: 10.1007/s10936-020-09720-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Alahmadi, A., Shank, C., and Foltz, A. (2018). Vocabulary learning strategies and vocabulary size: insights from educational level and learner styles. Vocab. Learn. Instruct. 7, 14–34. doi: 10.7820/vli.v07.1.alahmadi

CrossRef Full Text | Google Scholar

Alfadil, M. (2020). Effectiveness of virtual reality game in foreign language vocabulary acquisition. Comput. Educ. 153, 103893. doi: 10.1016/j.compedu.2020.103893

CrossRef Full Text | Google Scholar

Allen, M., Witt, P. L., and Wheeless, L. R. (2006). The role of teacher immediacy as a motivational factor in student learning: using meta-analysis to test a causal model. Commun. Educ. 55, 21–31. doi: 10.1080/03634520500343368

CrossRef Full Text | Google Scholar

AlSaif, A. (2011). Investigating Vocabulary Input and Explaining Vocabulary Uptake among EFL Learners in Saudi Arabia (PhD thesis). Swansea University. Available online at: https://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.678355

Google Scholar

Al-Sofi, B. B. M. A. (2020). Students' perceptions of the effectiveness of using smartphone applications in enhancing vocabulary acquisition. Int. J. Engl. Ling. 11, 110–124. doi: 10.5539/ijel.v11n1p110

CrossRef Full Text | Google Scholar

Ashiyan, Z., and Salehi, H. (2016). Impact of WhatsApp on learning and retention of collocation knowledge among Iranian EFL learners. Adv. Lang. Literary Stud. 7, 112–127. doi: 10.7575/aiac.alls.v.7n.5p.112

CrossRef Full Text | Google Scholar

Basal, A., Yilmaz, S., Tanriverdi, A., and Lutfiye, S. (2016). Effectiveness of mobile applications in vocabulary teaching. Contemp. Educ. Technol. 7, 47–59. doi: 10.30935/cedtech/6162

CrossRef Full Text | Google Scholar

Bensalem, E. (2018). The impact of WhatsApp on EFL students' vocabulary learning. Arab World Engl. J. 9, 23–38. doi: 10.24093/awej/vol9no1.2

CrossRef Full Text | Google Scholar

Blair, I. (2020). Mobile App Download and Usage Statistics 2020. Available online at: https://buildfire.com/app-statistics/ (accessed February 10, 2023).

Bouhnik, D., and Deshen, M. (2014). WhatsApp goes to school: mobile instant messaging between teachers and students. J. Inform. Technol. Educ. Res. 13, 217–231. doi: 10.28945/2051

CrossRef Full Text | Google Scholar

Brett, P. (2011). Students' experiences and engagement with SMS for learning in higher education. Innov. Educ. Teach. Int. 48, 137–147. doi: 10.1080/14703297.2011.564008

PubMed Abstract | CrossRef Full Text | Google Scholar

Bryer, T. A., and Zavattaro, S. M. (2011). Social media and public administration: theoretical dimensions and introduction to the symposium. Administ. Theory Praxis 33, 325–340. doi: 10.2753/ATP1084-1806330301

CrossRef Full Text | Google Scholar

Ceci, L. (2022a). Daily Mobile Message Volume of WhatsApp Messenger 2016-2020. Statista 2022. Available online at: https://www.statista.com/statistics/258743/daily-mobile-message-volume-of-whatsapp-messenger/ (accessed February 10, 2023).

Ceci, L. (2022b). WhatsApp: Monthly Usage in Selected Markets Q3 2021. Statista 2022. Available online at: https://www.statista.com/statistics/291540/mobile-internet-user-whatsapp/ (accessed February 10, 2023).

Çetinkaya, L., and Sütçü, S. S. (2018). The effects of Facebook and WhatsApp on success in English vocabulary instruction. J. Comput. Assist. Learn. 34, 504–514. doi: 10.1111/jcal.12255

CrossRef Full Text | Google Scholar

Church, K., and De Oliveira, R. (2013). “What's up with WhatsApp? Comparing mobile instant messaging behaviors with traditional SMS,” in Proceedings of the 15th International Conference on Human-Computer Interaction With Mobile Devices and Services (Athens), 352–361. doi: 10.1145/2493190.2493225

CrossRef Full Text | Google Scholar

Cook, J., Pachler, N., and Bradley, C. (2009). Bridging the gap? Mobile phones at the interface between informal and formal learning. J. Res. Center Educ. Technol. 4, 3–18.

Google Scholar

Dehghan, F., Rezvani, R., and Fazeli, S. (2017). Social networks and their effectiveness in learning foreign language vocabulary: a comparative study using WhatsApp. Call-Ej 18, 1–13.

Google Scholar

Desai, C. M., and Graves, S. J. (2006). Instruction via Instant Messaging reference: what's happening? Elect. Library 24, 174–189. doi: 10.1108/02640470610660369

CrossRef Full Text | Google Scholar

Fageeh, A. A. I. (2013). Effects of MALL applications on vocabulary acquisition and motivation. Arab World Engl. J. 4, 420–447.

Google Scholar

Fahruddin, F., Jana, P., Setiawan, J., Rochmat, S., Aman, A., and Yuliantri, R. D. A. (2022). Student perception of online learning media platform during the COVID-19 pandemic. J. Educ. Technol. 6, 42738. doi: 10.23887/jet.v6i1.42738

CrossRef Full Text | Google Scholar

Farmer, R. (2003). “Instant messaging collaborative tool or educator's nightmare,” in The North American Web-Based Learning Conference (NAWeb 2003).

Google Scholar

Hamzah, M. S. G., Kafipour, R., and Abdullah, S. K. (2009). Vocabulary learning strategies of Iranian undergraduate EFL students and its relation to their vocabulary size. Eur. J. Soc. Sci. 11, 39–50.

Google Scholar

Hill, J. B., Hill, C. M., and Sherman, D. (2007). Text messaging in an academic library: integrating SMS into digital reference. Refer. Libr. 47, 17–29. doi: 10.1300/J120v47n97_04

CrossRef Full Text | Google Scholar

Hsu, C. K., Hwang, G. J., Chang, Y. T., and Chang, C.-K. (2013). Effects of video caption modes on English listening comprehension and vocabulary acquisition using handheld devices. J. Educ. Technol. Soc. 16, 403–414.

Google Scholar

Hwang, W. Y., Huang, Y. M., and Wu, S.-Y. (2011). The effect of an MSN agent on learning community and achievement. Interact. Learn. Environ. 19, 413–432. doi: 10.1080/10494820903356809

CrossRef Full Text | Google Scholar

Jadhav, D., Bhutkar, G., and Mehta, V. (2013). “Usability evaluation of messenger applications for android phones using cognitive walkthrough,” in Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction (Bangalore), 9–18.

Google Scholar

Jafari, S., and Chalak, A. (2016). The role of WhatsApp in teaching vocabulary to Iranian EFL learners at junior high school. Engl. Lang. Teach. 9, 85–92. doi: 10.5539/elt.v9n8p85

CrossRef Full Text | Google Scholar

Jones, G., Edwards, G., and Reid, A. (2009). How can mobile SMS communication support and enhance a first year undergraduate learning environment? ALT-J 17, 201–218. doi: 10.1080/09687760903247625

CrossRef Full Text | Google Scholar

Klimova, B. (2021). Evaluating impact of mobile applications on EFL university learners' vocabulary learning–a review study. Procedia Comput. Sci. 184, 859–864. doi: 10.1016/j.procs.2021.03.108

CrossRef Full Text | Google Scholar

Koizumi, R., and In'nami, Y. (2013). Vocabulary knowledge and speaking proficiency among second language learners from novice to intermediate levels. J. Lang. Teach. Res. 4, 913. doi: 10.4304/jltr.4.5.900-913

CrossRef Full Text | Google Scholar

Mahdi, H. S. (2018). Effectiveness of mobile devices on vocabulary learning: a meta-analysis. J. Educ. Comput. Res. 56, 134–154. doi: 10.1177/0735633117698826

CrossRef Full Text | Google Scholar

Markett, C., Sánchez, I. A., Weber, S., and Tangney, B. (2006). Using short message service to encourage interactivity in the classroom. Comput. Educ. 46, 280–293. doi: 10.1016/j.compedu.2005.11.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Marko, M. (2023). 29+ Smartphone Usage Statistics: Around the World in 2022 LeftRonic 2022. Available online at: https://leftronic.com/blog/smartphone-usage-statistics/ (accessed March 10, 2023).

Nation, I. S. P. (2001). Learning Vocabulary in Another Language. Cambridge: Cambridge University Press.

Google Scholar

Rambe, P., and Bere, A. (2013). Using mobile instant messaging to leverage learner participation and transform pedagogy at a S Outh A Frican U Niversity of T Echnology. Br. J. Educ. Technol. 44, 544–561. doi: 10.1111/bjet.12057

CrossRef Full Text | Google Scholar

Rashtchi, M., and Yazdani, P. (2020). Intentional vocabulary learning via WhatsApp: does the type of input matter? Engl. Lang. Teach. Educ. J. 3, 118–132. doi: 10.12928/eltej.v3i2.2370

CrossRef Full Text | Google Scholar

Rau, P. L. P., Gao, Q., and Wu, L.-M. (2008). Using mobile communication technology in high school education: motivation, pressure, and learning performance. Comput. Educ. 50, 1–22. doi: 10.1016/j.compedu.2006.03.008

CrossRef Full Text | Google Scholar

Shen, Z. (2008). The roles of depth and breadth of vocabulary knowledge in EFL reading performance. Asian Soc. Sci. 4, 135–137. doi: 10.5539/ass.v4n12p135

CrossRef Full Text | Google Scholar

So, S. (2016). Mobile instant messaging support for teaching and learning in higher education. Inter. Higher Educ. 31, 32–42. doi: 10.1016/j.iheduc.2016.06.001

CrossRef Full Text | Google Scholar

Song, M., and Chen, L. (2017). A review on English vocabulary acquisition and teaching research in recent 30 years in China. Sci. J. Educ. 5, 174–180. doi: 10.11648/j.sjedu.20170504.18

CrossRef Full Text | Google Scholar

Stowe, W. A. Jr. (2013). Comparison of Learning Performance between Students Who Do and Students Who Do Not Use Mobile Technology-Based Activities. Denton, TX: University of North Texas.

Google Scholar

Taki, S., and Khazaei, S. (2011). Learning vocabulary via mobile phone: Persian EFL learners in focus. J. Lang. Teach. Res. 2, 1258. doi: 10.4304/jltr.2.6.1252-1258

CrossRef Full Text

Yearta, L. (2012). The Effect of Digital Word Study on Fifth Graders' Vocabulary Acquisition, Retention, and Motivation: A Mixed Methods Approach. Charlotte, NC: The University of North Carolina at Charlotte.

Google Scholar