Developing digital proficiencies of English learners in adult education

All around the globe, the development of digital skills is considered a priority because individuals are increasingly asked to interact with digital information. Digital literacy is required to communicate effectively and accomplish social and work-related tasks. Although there is a significant body of literature on the development of digital proficiencies in educational settings from elementary school to university, there is limited research with adults who are continuing their education later in life, usually outside of formal educational systems. The purpose of this paper is to report results from a study conducted with English learners in an adult education program in the U.S. The study evaluated the digital component of a new curriculum named CILIA-T (Content-Integrated Language Instruction for Adults with Technology Support), CILIA-T is a 16-module curriculum supporting adult learners to build their English skills, US History and Civics knowledge and digital proficiencies. In this study, researchers partnered with teachers to introduce part of the digital literacy content of the curriculum. There was a test given before and after the teaching. To determine the effectiveness of the digital literacy curriculum, the scores on the pre and post-tests and the duration to complete the pre and post-tests were analyzed. There were also learner and teacher surveys to evaluate the usability and utility of the curriculum. Main findings indicated that explicit instruction in digital literacy skills led to significant improvement across all participants. Teachers and learners found the digital literacy curriculum to be helpful and relevant.

Introduction

All around the globe, the development of learners' digital skills is considered a priority because in their lives, individuals are increasingly asked to interact with digital information (UNESCO, 2018, see for example Goal 4.2.2. “Youth and adults developing at least a minimum level of digital proficiency”). To comprehend digital materials, learners need sufficient vocabulary, background knowledge and higher-order thinking skills, similar to what is found with paper-based texts. However, digital materials also require additional skills, such as navigating the internet efficiently, evaluating the quality of internet sources and integrating materials across multiple modalities (picture, video, audio) and across multiple texts (Coiro and Dobler, 2007; Eckersley et al., 2023). Hence, educators believe that building learners' digital literacy skills is a growing need (Durgunoğlu et al., 2020; LAC, 2020). Although there is a significant body of literature on the development of digital proficiencies in educational settings from elementary school to university, there is a dearth of research in this area when it comes to adults who are continuing their education later in life, usually outside of formal educational systems (Eynon, 2021). In this paper, we briefly describe an adult education curriculum that includes a digital literacy component and report the findings from a study conducted to evaluate the effectiveness of this specific digital literacy component.

The digital literacy content is part of a larger 16-module curriculum called CILIA-T (Content-Integrated Language Instruction for Adults with Technology Support) that builds skills in US History, Civics and English for English language learners, with each module taking about a week of class time (6–8 h) to complete (Cary and Durgunoğlu, 2022). Because the whole curriculum includes digital activities interwoven throughout, Module 1 of the curriculum is dedicated to introducing the foundational digital skills that learners will continue to build throughout the curriculum. The digital component of CILIA-T also includes six short assessments to ascertain the digital literacy proficiencies of the learners before and after instruction.

The CILIA-T curriculum team determined which digital skills to focus on throughout the modules through a comprehensive process that including reviewing existing digital literacy standards and frameworks (Vanek, 2017; Northstar Digital Literacy Standards, 2024). The team identified the specific digital literacy proficiencies that learners would need to use throughout the curriculum and determined the foundational digital literacy skills necessary for learners to effectively engage with the digital tools used in the curriculum.

Through this process, we identified six tools/tasks that learners will use throughout the curriculum. These include Quizlet (vocabulary practice), Gmail and WhatsApp (accessing homework tasks and communicating with teacher and classmates), Zoom (accessing content remotely), Smartphone Apps (accessing apps and internet via a mobile device), and Finding Information Online (navigating websites, using browsers). However, concurring with Coiro (2011, 2021), we assume that digital literacy includes additional proficiencies that go beyond skillful use of tools. Hence, in CILIA-T, we focused on six digital competencies : (1) Functional Skills/Navigation Skills ((2) Finding and Evaluating Digital Information (3) Responsible Digital Citizenship (4) Technology Troubleshooting (5) Comprehension and Integration (6) Communication, Collaboration, and Creation.

In this paper, we report the findings from a study in which we evaluated this digitally integrated curriculum by using one of the assessments, titled “Finding information online” both as a pre and post- test, and delivering the relevant digital lesson in-between the two assessments. In the full curriculum, Module 1 (the foundational digital skill building lesson) is taught in about 6–8 h, with additional time set aside for all assessments. Because we conducted this digital pilot study in one class period of about 2.5 h, we evaluated only one of the assessments and its relevant content. Table 1 summarizes the 14 skills that were included in this assessment. The items labeled as “studied” were explicitly taught in class. Those labeled as “non-studied” were not discussed in class, but at the end of the study, those non-studied materials on the test were provided to the learners as a text with screenshots. This way, the learners would know about all of the topics that were asked on the test.

Table 1

Table 1. Items on the assessment used as pre- and post-test and the error rates on each item.

In this paper, we address the following research questions:

1. How effective was the lesson (content and the activities) on building foundational digital skills and adult learners' understanding of the concepts?

2. How did adult learners and the teachers view the instructional content?

3. How were the participants' background characteristics related to the development of their digital proficiencies?

Methods

Participants

There were 29 participants from three different high-intermediate/low-advanced ESL classes at three different sites in a large U.S. city. In all three classes, the students had access to tablets/laptops. All students knew how to log on to their school accounts and they were familiar with accessing the online English teaching programs previously introduced to them by their teacher. However, they were not doing any independent internet searches as part of their regular instruction. Of the 26 participants reporting their country of origin, 8 were from Central and South America; 8 from Africa; 12 from Europe; 3 from Asia. Of the 27 participants reporting gender, 21 were women (72%). Of the 27 reporting their race/ethnic background, 41% were Black, 24% Hispanic, 17% white and 7% Asian. On average, the participants had completed 11 years of education (SD = 3.99). The average age was 25 (SD = 9.2). The age of arrival to the U.S. was 30 (SD = 10.32). However, 70% had been living in the U.S. for 5 years or less. On the demographic form, participants indicated how often they used digital tools. A total of 93% reported using the internet 3–7 days per week. Most of this was on their phone, as 89% reported that they used a smartphone 3–7 days per week, whereas only 61% reported using a computer 3–7 days/week.

Materials

Each student had a packet of materials which consisted of recording sheets for the pre and post-tests, the text for the lesson “Accessing the Internet” and a brief survey. The text was about 10 paragraphs long, with large visuals. It also included classroom activities labeled “Let's Chat” and “Let's Practice.”

The participants completed a survey at the end of the study which asked them to report the following background information: Age, age at arrival to the U.S., country of origin, gender, race/ethnic background, and number of years of schooling. Participants reported how often (in 1 week) they (i) use smartphones, (ii) use a computer and (iii) access the internet, by checking one of the alternatives: Every day / 5-6 days / 3-4 days / 1-2 days / None. They also rated their familiarity with the topic taught in the lesson by marking one of the four alternatives: I knew a lot/I knew some/I knew little/ I did not know anything. Finally, participants rated on a 5-point scale how easy they found the pre-test and the post-test, and how clear and how helpful the lesson was.

Teachers also completed a survey about the lesson. They were asked six open-ended questions:

(1) How relevant and usable was the material for adult learners? (2) How clear was the material, especially considering the comprehension proficiencies of English learners? (3) How useful were the “Let's Chat” discussion activities? (4) How useful were the “Let's Practice” activities? (5) Any suggestions on making the content better? (6) Feedback on the usability and clarity of the additional material [given at the end] and their suggestions for improving it.

Procedure

The research was introduced and learners listened to and read along the information form and asked questions before they consented to participate in the study. Those who decided not to participate in the study could participate fully in class and do the pre- and post-tests, but they would not share their data. One class was held in a hyflex format with a few students attending class from home while most of the students were in the classroom. For students at home, the full packet of the materials (information sheet, reading materials, pre- and post-test recording sheets and the survey) was mailed to their homes a week before the study, as this was the usual mechanism for them to receive the classroom materials. Those students participated in class via Zoom on their phones or tablets, but did not access or download any course materials from any links. The packets also included an envelope addressed to one of the researchers for those individuals to send their data if they wished (none of the six hyflex students participating from home sent in their materials). Across the three sites, among the students attending the course in-person, only one decided not to participate (hence we use the word “students” when referring to the whole class, but use “participants” when referring to those who agreed to be in the study and shared their data).

After the informed consent process, the students were asked to get their tablets/laptops from their cabinet and log on with their student password. They were also given the packets with the reading materials, pre- and post-test recording sheets and the survey. Then the researcher put a tinyurl address on the board and asked the learners to type the address in a browser. The students received help in typing this address. This address opened the pre-test. Before the pre-test, there was a 2-min NorthStar video explaining the mechanics of the assessment, such as how to play the audio for a question, where to click to respond and how to go back to any unanswered questions or to change an answer and how to submit. After this introduction, if they had questions, those were answered and each student started the assessment. The classroom teacher and the researcher walked around and helped if there was a problem, such as a frozen machine, but they did not help with any test items. If the students asked for help regarding a test item, they were told that they could skip a question by pressing the “I don't know” button. The responses were collected electronically, however as a precaution against data loss, students also copied what was on the final results screen (their test ID number and score) onto a pre-test recording paper.

Once all students were finished with the pre-test, the lesson was taught (discussed in the next section), followed by the same test, now given as a post-test. On the post-test, the students could skip the introductory video if they chose to do so. After the post-test, students again recorded their results on the recording sheet and then completed the survey collecting their demographic information and views about the digital activities. At this point, those who wanted to share their data handed in the completed forms and received $15 compensation. Afterwards, all students received the debriefing form which discussed topics that were not taught in the lesson, but were on the test (non-studied items). The whole study took about 2–2.5 h.

The lesson

The digital content was taught by teachers in two classes and by one of the authors in the third class. The lesson started with the following questions written on the board, with the answers discussed as a whole group: “What have you used the internet for in the past? What can you use the internet for? What do you want to use the internet for?” The goal was to activate the background knowledge of the learners and get them to think about the topic of the lesson. Students were told that they were going to learn some basic internet skills that they were asked on the pre-test, and then take the test again. For the actual teaching of the content, the students took out the article “Accessing the Internet” and followed along as the teacher read paragraphs (or 2–3 sentences apiece) aloud to the class, stopping after each section to ask the “Let's Chat” questions about that section and call on various students to answer and to guide students through the “Let's Practice” steps for each section.

The lesson had the following parts: (1) Text and visuals describing different ways to connect to the internet (2) Let's Chat activity where the students discuss how they usually connect to the internet (3) Text defining and giving examples of the terms browser, website and (hyper)link (4) Let's Practice activity, in which the students examine the browser(s) on their devices, click on a browser and type the course website in the address bar (5) Text discussing how to determine the reliability of websites (6) Let's Chat activity determining the reliability of three sites for someone interested in learning more about diabetes (7) Text on navigating websites and examining links (8) Let's Chat activity, in which the students examine the class website and note the items linked to that website and how to go back to the homepage after examining one of the linked pages (9) Let's Practice activity in which students learn how to bookmark favorites (10) Text on internet safety (11) Let's Chat activity on the rationale for not saving one's password on public computers.

Results and discussion

Question 1: effectiveness of the curriculum

Three different outcomes were used to determine the effectiveness of the curriculum: (a) The percent correct responses on pre- vs. post-tests, (b) time it took to complete the pre- vs. post-tests (c) reported ease of pre vs. post-tests on the survey. The t-tests comparing pre and post-test values are summarized in Table 2. All three outcome measures showed significant positive development. Participants answered more questions correctly, and found the post-test to be easier. The post-tests were completed faster (of course, some participants skipped the introductory video on the post-test, thus finishing the post-test more quickly. We could not separate the video watching times from the test completion times. However, learning how to complete an online test is also a digital proficiency, so we can view the post-test durations as also including this development).

Table 2

Table 2. Mean, standard deviation (SD), and statistics on pre and post outcome measures.

As an another indicator of the lesson's effectiveness, pre-test and post-test error percentages were compared for studied vs. non-studied items. A 2x2 repeated measures ANOVA (Time of Test: Pre and Post X Item Type: Studied and Nonstudied) was conducted on the percent errors that are summarized in Table 3. Both Time of Test and Item Type were significant, F_{(1, 28)} =15.31, η² = 0.35 and F_{(1, 28)} = 5.94, η² = 0.18, respectively, indicating that error rates were lower on post-tests and rates werelower on studied items. The interaction between the two variables did not quite reach significance, F_{(1, 28)} = 2.27, p = 0.14, although the studied items showed a larger drop in error rates. Overall, the digital lesson was effective in improving the participants' performance as assessed by multiple types of outcomes.

Table 3

Table 3. Percent errors on studied and non-studied items on pre and post-tests.

Question 2: content evaluation

The whole teacher survey and two questions from the participant survey were analyzed to address this topic. When rating the clarity of the lesson, 35% of the participants said the lesson was okay and 61% said the lesson was clear or very clear. Only one participant (4%) found the lesson to be unclear. In terms of helpfulness, 40% found the lesson okay, whereas 60% found it helpful or very helpful. Nobody found the lesson to be unhelpful.

All three teachers reported that the lessons were clear, relevant and useful. Teachers all agreed that this material was appropriate for the advanced English learners. One teacher suggested using bullet points rather than paragraphs of text and also using arrows to better highlight where to click. Another suggested enlarging the pictures accompanying the text. These results indicated that the program was perceived to be usable and helpful by both students and teachers.

Question 3: digital proficiency as relating to learner characteristics

Correlations were computed between several background variables and outcome measures. As shown in Table 4, post-test scores were significantly related to participants' existing proficiencies as there were significant correlations of post-test scores with pre-test scores (0.56) familiarity of the topic (0.54), reported internet use (0.40) and education level (0.55). Post-test scores also correlated with how clear (0.64) and helpful (0.49) the lesson was rated. Internet use was not related to age, education or pre-test scores, however age was related to how helpful the lesson was perceived by the participants. Younger participants found the lesson to be more helpful. Individuals who found the lesson clear also found them helpful (0.58). The correlation data indicated that although the curriculum led to significant improvement across all participants (Question 1), those with relatively higher levels of education, experience and familiarity tended to have higher post-test scores. The correlations indicate that for adult learners, existing skills play an important role in facilitating the development of their digital proficiencies.

Table 4

Table 4. The correlations between learner characteristics and pre- and post-test scores.

Conclusion

This digital literacy lesson yielded significant improvement in adult learners and was received positively by both teachers and learners and hence could be a model for those developing digital literacy curricula in adult education programs. Not surprisingly, those with relatively higher levels of education, experience and familiarity tended to have higher post-test scores. This pattern indicates that increased class time for differentiation of instruction and allowing for more opportunities to practice the digital skills could benefit lower-level learners with beginning digital literacy skills.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by the University of Minnesota Institutional Review Board. The studies were conducted in accordance with the local legislation and institutional requirements. The ethics committee/institutional review board waived the requirement of written informed consent for participation from the participants or the participants' legal guardians/next of kin because as a low-risk educational study a signature was not required. However participants still listened to and read along the consent form and kept a copy of the form themselves to contact IRB if they want. The only waiver was for a signature. There was a written consent process but no signature was collected.

Author contributions

LH: Investigation, Resources, Software, Writing – review & editing. AD: Conceptualization, Data curation, Funding acquisition, Investigation, Methodology, Project administration, Writing – original draft. JS: Data curation, Software, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant 305N210032 to University of Minnesota (PI: AD).

Acknowledgments

We thank Jesse Morton, Theresa Sladek and teachers Thavrak Hay, Ifeng Hsu and Christian Peterson for their help.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Author disclaimer

The opinions expressed are those of the authors and do not represent views of the Institute of Education Sciences or the U.S. Department of Education.

References

Cary, E., and Durgunoğlu, A. Y. (2022). Civics/History Curriculum: An Introduction to the CILIA-T Curriculum Project. Available online at: https://atlasabe.org/news/civics-history-curriculum-an-introduction-to-the-cilia-t-curriculum-project/ (accessed April 5, 2024).

Google Scholar

Coiro, J. (2011). Predicting reading comprehension on the internet: Contributions of offline reading skills, online reading skills, and prior knowledge. J. Liter. Res. 43, 352–392. doi: 10.1177/1086296X11421979

Crossref Full Text | Google Scholar

Coiro, J. (2021). Toward a multifaceted heuristic of digital reading to inform assessment, research, practice, and policy. Reading Res. Q. 56, 9–31. doi: 10.1002/rrq.302

Crossref Full Text | Google Scholar

Coiro, J., and Dobler, E. (2007). Exploring the online reading comprehension strategies used by sixth-grade skilled readers to search for and locate information on the internet. Reading Res. Q. 42, 214–257. doi: 10.1598/RRQ.42.2.2

Crossref Full Text | Google Scholar

Durgunoğlu, A. Y., Gençay, H., Cantürk, M., and Kuscul, G. H. (2020). “Cognitive and sociocultural dimensions of adult literacy and integrating technology in adult education,” in Wiley Handbook of Adult Literacy, ed D. Perin (New York, NY: Wiley), 263–282.

Google Scholar

Eckersley, N., Hauge, L., Durgunoğlu, A. Y., and Cary, E. (2023). CILIA-T Curriculum Project: Developing Our Digital Skills Roadmap. Available online at: https://atlasabe.org/news/cilia-t-curriculum-project-developing-our-digital-skills-roadmap/ (accessed April 5, 2024).

Google Scholar

Eynon, R. (2021). Becoming digitally literate: reinstating an educational lens to digital skills policies for adults. Br. Educ. Res. J. 47, 146–162. doi: 10.1002/berj.3686

Crossref Full Text | Google Scholar

LAC (2020). The Impact of COVID-19 on NYC Adult Literacy Programs and Students: Survey Findings from the Literacy Assistance Center. Available online at: https://bit.ly/COVID19ImpactAdultEdu (accessed April 5, 2024).

Google Scholar

Northstar Digital Literacy Standards (2024). Available online at: https://www.google.com/url?q=https://www.digitalliteracyassessment.org/features%23standardsandsa=Dandsource=docsandust=1711577377775114andusg=AOvVaw3WSnzMadnkjMMJlbCLtg_G

Google Scholar

UNESCO (2018). A Global Framework of Reference on Digital Literacy Skills for Indicator 4, 4.2. Available online at: https://uis.unesco.org/sites/default/files/documents/ip51-global-framework-reference-digital-literacy-skills-2018-en.pdf (accessed April 5, 2024).

Google Scholar

Vanek (2017). Digital Literacy. Available online at: https://lincs.ed.gov/sites/default/files/TSTMDigitalLiteracyBrief-508.pdf (accessed April 5, 2024).

Google Scholar