Teaching Human-Computer Interaction Modules—And Then Came COVID-19

In teaching Human-Computer Interaction at university level, it has always been beneficial to explain the related theory and engage students in a practical way, whether individually or in groups. And then came COVID-19. Face-to-face classes were replaced by emergency remote teaching methods. Students became student numbers in cyber space. The danger became real to convert back to the traditional way of presenting lectures, namely a lecturer doing all the talking and the students being the passive audience. This paper describes how the author had to adapt and innovate in terms of teaching Human-Computer Interaction modules to university students in a practical way during the COVID-19 pandemic. Frequent online quizzes, audio messages, online group discussion, smaller topic-dedicated practical activities, and webinars encouraging student participation, were employed. Instead of having access to eye-tracking technology in a usability laboratory, students had to innovate for usability evaluation assignments by employing observation, think-aloud protocols, and performance and self-reported metrics as data gathering methods. The laboratory had to be replaced by COVID-compliant places of residence. The outcomes of adapting previously-used teaching methods and inventing new ways to encourage student participation, were surprisingly positive. An additional advantage was that many of these methods turned out to be so successful that their application could be continued and extended to post-pandemic times for a blended learning approach to further enrich Human-Computer Interaction teaching.

Introduction

According to the Stirling Institute of Australia (2018), although learning styles differ, practical learning is the one type of learning that benefits most students. An improved skills set, increased understanding, the creation of a deeper impact and better knowledge retention are but a few of the advantages of practice-based learning.

In teaching Human-Computer Interaction (HCI) at university level, it has always been beneficial to explain the related theory in a practical way, as well as to engage students in this practical application, whether individually or in groups. According to Churchill et al. (2013), the relationship between theory and practice when teaching HCI is an issue that has often been raised. Although a solid theoretical basis is needed, hands-on projects and practical experience are considered important. Oleson mentions in Bits and Behavior (2020) that although interaction design is hard to teach and hard to learn, these students should ideally come out of these classes with enough proficiency to avoid common pitfalls in the software that they design. This proficiency could be enhanced by approaches to assist students to practically apply theoretical knowledge. Examples of these approaches include the student-use of mobile devices in class where they could reinforce their understanding of certain principles or topics with a hands-on approach, class quizzes, incorporating videos in lectures, as well as encouraging group discussions.

During the author’s teaching career, the practical application of knowledge when teaching HCI also extended to students conducting usability evaluations in a usability laboratory on campus. Here they had access to specialized technology, such as eye-tracking. The benefits of this practical approach have been numerous, including better understanding of the theory, better results, enthusiasm towards the module, as well as frequent “light bulb moments”.

And then came COVID-19. Face-to-face classes were replaced overnight by emergency remote teaching (ERT) methods. Students became student numbers in cyber space. Using the mobile devices and the usability laboratory on campus was not possible any more. Instead of making use of eye-tracking technology, students had to revert to only using observation, think-aloud protocols, performance and self-reported metrics as data gathering methods in their usability evaluation assignments. The usability laboratory had to be replaced by COVID-compliant places of residence. The danger became real to convert back to the traditional way of presenting lectures, namely a lecturer doing all the talking and the students being the passive audience.

Higher Education Institutions (HEIs) across the globe faced multiple challenges due to the pandemic. Various studies have been performed recently to investigate these challenges—faced by both staff and students. Examples include the Global Survey of College and University Leadership from the International Association of University Presidents and Santander Universidades (O’Malley, 2020), a study undertaken in Norway (Langford & Damsa, 2020), another in Cyprus (Souleles et al., 2021), and a study at the University of the West Indies (Thurab-Nkhosi et al., 2021). Although most of these studies identified access to technology and the maintenance of academic standards as challenges, faculty training in ERT was highlighted as a serious need. With such training (or professional development in this area) not generally being part of an academic’s pre Covid-19 armor, many an academic had to adapt and innovate almost overnight as a result of the pandemic.

This curriculum instructions pedagogy (CIP) paper describes how the author had to adapt and innovate in terms of teaching Human-Computer Interaction modules to university students in a practical way during the COVID-19 pandemic. The paper is organised in the following way: the pedagogical foundation is first addressed by defining HCI and presenting an overview of the two HCI modules discussed in this paper. This includes a brief glimpse of the differences between before and amidst COVID-19. Then the learning environment is discussed by explaining the South African COVID-19 context and the teaching setting and methods. This involves the methods employed during the COVID-19 pandemic to adapt learning materials and communications methods, to teach theory classes, and to handle practical assignments and assessments. The paper is concluded with insight into the practical applications and lessons learnt during this trying, but unique experience.

In the rest of the paper the term “author” and will sometimes be replaced by the term “lecturer” in those instances where the context is that of teaching HCI.

Pedagogical Foundation

Human-Computer Interaction

Sharp et al. (2019) describes Human-Computer Interaction (HCI) as the study of computer systems from the user’s point of view. It falls under the umbrella term of Interaction Design. While HCI has a narrower focus on the design and usability of computing systems, the scope of interaction design is broader, concerned with theory, research and practice of designing the user experience for all manner of technologies, systems, and products.

Human-Computer Interaction Modules

In the tertiary institution where the author is lecturing, HCI is taught as both a second year and Honours level module in the Department of Computer Science and Informatics. The author is responsible for teaching both of these modules.

Learning Objectives and Outcomes

The aim of the second year module is to introduce students to the fundamental and exciting discipline of HCI in order to understand the designs that people need, to distinguish between poorly and well-designed user interfaces, and to learn the basics of usability testing. The students are, therefore, introduced to topics like usability, the user experience (UX), design principles, conceptualization, cognitive and emotional aspects, interface and screen design choices, and patterns for prototyping. They are briefly introduced to data gathering and the evaluation of prototypes and final products.

Therefore, after successful completion of the module, it is expected that the second year students will:

• Display knowledge of the basic principles of Interaction Design

• Identify usability goals and user experience goals for proposed interactive systems

• Redesign poorly designed user interfaces

• Distinguish between and select appropriate interface types

• Determine user needs and convert them into requirements in order to ensure that the interactive product helps the user to perform his/her tasks in an effective, efficient, and enjoyable manner

• Display and apply knowledge on the evaluation of a user interface (interactive product) in terms of usability.

On Honours level, the focus is mainly on usability engineering and evaluation. In this case, the students need to be able to do the following after completion of the module:

• Explain the issues involved in the evaluation of interactive systems

• Identify the different types of metrics used in evaluating the user experience

• Plan and design a usability study

• Conduct a usability study, making use of the usability laboratory and its equipment in the department (if access to campus is possible).

For both these modules it is crucial that the students learn how to apply in practice the theoretical principles.

Pre-COVID-19

Pre-COVID-19, modules were presented in face-to-face mode at our tertiary institution. Although aspects of blended learning were incorporated on a small scale, it was more the exception than the rule.

Undergraduate theory classes were presented on campus twice or three times a week, while a 3-h practical took place in one of the computer laboratories. IPads were incorporated as part of the theory classes where students could participate in class by means of online quizzes, or by doing individual or group activities to practice the taught skills. This method had the additional advantage of providing the lecturer with immediate feedback to see whether the students are on board or not. Periodical practical assignments included the evaluation of existing systems according to usability and user experience goals, identifying various interaction and interface types in existing systems, and improving badly designed screens or questionnaires by applying navigational and screen design patterns.

The Honours students attended one 2-h contact session on campus per week, while practical assignments could be done in their own time by making use of the departmental or the usability laboratory. These students were expected to plan and perform a full-fledged usability evaluation in the usability laboratory, recruiting real users, and using various technologies, including the eye tracker.

Amidst COVID-19

As we received the email on March 18, 2020 that the university will be locked down basically with immediate effect, both staff and students were totally unprepared for what lied ahead. The government announced that all post school training institutions would have an early recess, starting that day, to minimise risk of COVID-19 to all its staff and students. As the national lockdown was declared to last for 3 weeks, the general idea was that we would be able to return to campus after 3 weeks. The author left campus with her laptop, a few files and (fortunately) remembered to grab the Honours text book (the module that she was teaching at the time). Little was she to know that staff and students would only be able to fully return to campus in the following year.

This was the start of lecturing staff’s exposure to the ERT environment. The recess period was to be used to adapt all plans, schedules, and learning materials from face-to-face to “online” mode, as well as to comply with newly developed University and Faculty contingency and catch-up plans. The university’s Division for Teaching and Learning immediately started to compose material concerning the extended use of the learning management system (LMS) that were applicable and useful to both staff and students. They also started presenting online workshops to assist staff in how to apply this well-organized digital learning platform (ispring, 2021), especially in terms of the LMS functions that have seldom been used before.

Learning Environment

In order to understand the challenges of adapting lecturing procedures, lecturing practices and learning materials to emergency remote mode while preserving quality of teaching, is it important to understand the learning environment of South African students during COVID-19.

South African COVID-19 Context

With the university relevant to this paper being situated in the central province of the country, many of its students come from rural areas. Being a residential university, these students reside either on campus in university residences, or in student housing in nearby suburbs. With the national lockdown, these students had to return home. Having to study from home implied for some the reality of living in an area that has not yet received access to electricity. Additionally, it was during this time that the national energy provider, Eskom, incorporated frequent national load shedding periods as a result of not being able to supply in the country’s energy demands.

If access to the Internet was possible, the bandwidth was often not adequate for a student’s needs. With data costs and financial difficulties that many students and their families experienced, internet access was problematic. The majority of students generally make use of computer laboratories on campus to do their practical assignments. A large number of them do not own their own computers or laptops, therefore, with moving back home even computer access was not a given.

In order to attempt to address these issues, the university launched a campaign to obtain laptops that could be lent to students to work on during the lockdown period. First of all, all the students needed to be contacted in order to determine the need for computers. Then they were required to supply their physical addresses where couriers could deliver the laptops and sign agreements. However, the negative COVID-19 economic implications caused a shortage of computer components in the country. All these aspects delayed the process considerably.

To address the issue of inadequate bandwidth and data, negotiations by Universities South Africa with national mobile provides were undertaken. Eventually an agreement was reached that allowed for a selection of academic websites to be zero-rated, including the LMS used by our university. This implied that they were exempted from data fees by the four major mobile providers in the country. This also addressed the communications problem, as emails sent and received via the LMS would be free of charge. However, one has to bear in mind that all these provisions were not yet in place by the time the classes resumed after the recess. They all realized over time.

Teaching Setting and Methods

The Coronavirus caused a shift that forced the adoption of new ways of teaching and learning on both instructors and learners (Kristen et al., 2021). This gave way to emergency remote education that is referred to as ERT, or emergency remote learning (ERL). On the Educause Review (2020) website, (Hodges et al., 2020) define ERT as “a temporary shift of instructional delivery to an alternate delivery mode due to crisis circumstances”. It involves the use of “fully remote teaching solutions for instruction or education that would otherwise be delivered face-to-face or as blended or hybrid courses and that will return to that format once the crisis or emergency has abated”. Both positive and negative aspects of ERT and ERL have been reported. The positives include that students learnt new skills through associated in-class technology, more freedom for breaks and learning at a slower pace, better problem solving and more family time. In contrast, the negatives include the stifling of learning effectiveness by technical issues, isolation that can have a negative effect on mental health, and increased responsibility at home (Beekman, 2021).

In comparison to ERT and ERL, online teaching allows for community building among students and with the lecturer, and provide opportunities for learners to engage with learning material through practice, repetition, real-world context and feedback. Online learning also provides a safe environment for all leaners, thus addressing accessibility (Schlesselman, 2020).

ERT required that learning material, as well as communication methods, be adapted to cater for low bandwidth. Lecturers also had to provide for a variety of (often new) ways to present content. Theory classes had to be presented online, while alternative measures had to be devised to ensure that students still had the opportunity to be taught how to apply the theory in a practical way. Lastly, assessments had to be re-invented and re-designed.

Each of these aspects will now be discussed in terms of the second year and Honours HCI modules presented by the author. As many of the principles applied were applicable to both modules, a distinction will only be made when appropriate.

Learning Material and Communication

One of the first tasks was to adapt PowerPoint slides. Many students did not have their text books with them, therefore the slides had to be extended to additionally serve as study guides. To ensure that students with access to low bandwidth would still be able to access the slides, all unnecessary aesthetics, e.g. images, had to be removed. Images that were deemed necessary, had to be compressed. In a practical module like HCI, this posed a challenge, as many of the explanations provided previously were done with the aid of images or sketches. Heading and text typeface sizes in slides also had to be reduced. After all these issues were addressed, the slides were compressed and converted to pdf-files.

In order to ensure that students were not left in the dark, especially in the beginning of the lockdown period where a lot of uncertainty prevailed, communication with students had to be taken to a next level. The author started by preparing an audio message where the plans and procedures for the semester were explained. Students were also assured of continuous support, of which weekly audio messages and schedules being made available each Monday morning before 09:00, formed an integral part. Email communication and lecturer announcements took place from within the LMS. A WhatsApp group was created for the class, where urgent messages could be posted.

Theory Classes

Theory classes were presented once a week via webinars held from within the LMS. This is where the challenge of class participation and practical applications became tricky. IPads could no longer be used in class. Neither could face-to-face group activities.

To compensate for the loss of in-class quizzes, compulsory weekly quizzes were set up in the LMS. These quizzes mainly focused on the theoretical part of the work and tested whether students were following and understood the basics.

To additionally encourage class participation, the lecturer sporadically made use of online polls during these online classes. The students had to indicate if they agreed with a certain statement (yes/no response), or they were provided with a multiple choice question that they immediately had to answer. The poll responses were displayed in real time, which meant that the students could rate or compare their answers to those given by the rest of class. After closing the poll, the lecturer then discussed the responses.

Face-to-face in-class group activities were replaced by smaller group discussions during the online classes. A question or statement was posed by the lecturer who subsequently divided the class into smaller groups. The groups were instructed to each appoint a spokesperson for feedback afterwards. While these groups broke away for the discussions, the lecturer was able to “visit” each of these groups in turn to get insight into their discussions. After a predetermined time period, the class was again united and some of the groups were asked to provide feedback. Having a spokesperson appointed by the groups really helped and motivated participation. It seemed to be a confidence booster, as the lecturer’s past experience was that when a question was posed to the whole class, nobody often responded.

Practical Assignments

Due to the connectivity challenges that some of the students faced, deadlines for completing assignments had to be extended.

The second year HCI students had to complete four assignments during the semester that were intended to teach them to apply theory into practice. These included an assignment where a badly designed website had to be selected and evaluated based on the conformance (or not) to usability goals, user experience goals and design principles. To address data gathering, a badly-designed questionnaire had to be redesigned by adhering to good questionnaire design principles. For screen design, an assignment consisting of various screens (websites and mobile screens) were presented to students where they had to either identify or apply navigation, general layout, mobile layout or visual design patterns, or apply them. The final assignment required them to suggest various evaluation strategies for scenarios presented. These assignments were released to the students via the LMS, generally providing space for answering the questions on the question paper itself. They then had to upload their answers via the LMS as well before the due date.

Pre-COVID-19, the Honours HCI students were required to complete a large semester assignment that involved the planning and conducting of a usability study where real users had to perform tasks on the system in question in our Department’s usability laboratory. The use of the eye-tracker in the laboratory was compulsory. However, with the COVID-19 restrictions the students were firstly not able to get access to campus (and therefore to the laboratory), nor were they allowed to approach prospective test participants. A re-invention of the semester assignments was necessary.

The assignment was changed to require students to plan the usability test without incorporating the eye-tracker. The usability evaluation had to be performed in their place of residence (with adhering to all the COVID-19 sanitary requirements) and they had to use the people around them who they had access to, as test participants. As far as data gathering methods were concerned, they had to rely only on observation, the think-aloud or retrospective think-aloud protocol, and asking users (questionnaires and interviews). The lecturer acted as expert to conduct expert evaluations. The assignment was broken down into modules with interim deadlines. The students were then able to send their work-in-progress to the lecturer before each due date and would receive initial feedback on the part of the work done. This served as quality control, but also as a motivational factor for students to keep their momentum going as far as assignment progress was concerned.

Performance metrics (task success, time on task, number of errors, and efficiency), self-reported metrics (various types of questionnaires and interviews) and issue-based metrics could be incorporated in the usability tests, as no special equipment was necessary to do this. Students were also required to combine metric results in order to obtain a single usability score (Tullis and Albert, 2013)

Assessments

During these times, the university followed a “no student left behind” policy. This entailed that no assessments where marks counted towards the semester or final mark, were to take place during the first 6 weeks of lock down. This allowed students time to address connectivity and computer problems. This policy also required lecturers to provide longer time periods for answering or submitting assessments, as well as to provide multiple assessment opportunities for those who missed tests and assessment deadlines due to sickness, family trauma, or even load shedding. It placed quite an additional work load on staff, as it could involve the provision of up to three additional tests or examination papers.

Major semester assessments had to be prepared online, which posed new challenges due to the fact that students now had access to all their study material during tests. To counteract, the tests were transformed to an assignment format, and the type of questions posed were mostly focused on the application of knowledge. Case studies were used to sketch scenarios to which solutions needed to be found.

As mentioned by Adedoyin and Soykan (2020), summative assessments during ERT bring along the challenge of eliminating cheating and plagiarism, which was addressed by the author by requiring the students to submit their assignments through Turnitin (turnitin, n.d.), a plagiarism checker integrated with the LMS used by the university.

Conclusion

Practical Implications

Adapting the lecturing of HCI modules at tertiary level had practical implications. Firstly, class attendance at our university is compulsory. During the pandemic, class attendance (for online real-time classes) tended to be poor. This came despite the fact that the online classes presented in the LMS allowed for attendance lists to be pulled after class. As very little control existed over class attendance with all the possible problems that students could potentially experience (i.e., connectivity, illness, hospitalization, load shedding), some students made use of this opportunity not to attend classes. However, as classes were recorded and these recording links being available to students, one could not ultimately determine whether students did in fact attend these classes offline.

Communication with students needed a conscious effort. It needed to take place regularly and in such a way that students would never be unsure of what was expected of them. Collaboration among students increased. It seemed to be easier for them to communicate electronically with co-students compared to face-to-face communication. The discussion board in the LMS provided a useful platform for this collaboration to take place.

The author needed to stand back and consider HCI from a new perspective. It required creativity and innovation to keep classes interesting, practical, and to adapt assignments and assessments to ensure practical application, all while attempting to adhere to the expected standard of teaching.

As far as the student results were concerned, the second year average stayed similar compared to previous years, but the average of the Honours students increased with 3%. This was quite a heartwarming realization amidst the extremely trying circumstances. And as far as all the students who did their part were concerned, nobody was, indeed, left behind.

Lessons Learnt

In retrospect, very important lessons were learnt through this COVID-19 teaching experience. These lessons include realizing the importance of communication with students and collaboration among students, the importance of good and extensive planning of a module, the advantages of presenting the same content in multiple ways, and the advantages of blended learning. In terms of the students themselves, the experience provided insight into the real-life circumstances of students at home and those aspects that one often takes for granted, like access to electricity, computers, and the internet, which are not always a given. The author was also reminded of the privilege of being on campus to gain access to all its facilities, for both staff and students.

Contributions

This paper described the experiences and insights gained while teaching HCI during the COVID-19 pandemic. Despite the fact that face-to-face teaching might in future again become a generally applied mode of teaching (post-COVID-19), blended learning can assist HCI lecturers in providing opportunities to expose students to the same content in different ways. Incorporating compulsory online quizzes as a way to ensure that students prepare before attending a class, for example, could serve as student motivation and ultimately, better results. This is in line with the findings of Adedoyin and Soykan (2020). Additionally, incorporating audio messages, online group discussions, smaller topic-dedicated practical activities and webinars, can improve student participation.

HCI students can be encouraged to be innovative in conducting usability evaluations as part of practical assignments, especially in a third-world context where access to top-of-the range technology is not a given. Instead of incorporating, for example, eye-tracking technology in a usability study, valid evaluation results can also be obtained by employing observation, think-aloud protocols, and performance and self-reported metrics as data gathering methods. As the laboratory had to be replaced by COVID-compliant places of residence, it showed that usability evaluation could be performed in a variety of alternative locations, as long as the evaluators adhere to the basics of setting up a controlled environment.

Although challenging, these experiences pathed the way to being perceptible to possibilities for innovative teaching strategies in future.

Data Availability Statement

The original contributions presented in the study are included in the article. Further inquiries can be directed to LdeW (DWetL@ufs.ac.za).

Author Contributions

As I am the only author of this paper, my contribution was 100%. LdW.

Conflict of Interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The author would like to thank the Centre of Teaching and Learning at the tertiary institution for their efforts in assisting staff and students during the pandemic. A specific expression of gratitude to Jani van der Merwe, a senior officer and learning designer of the LMS, for all her unprecedented help and guidance through difficult times.

References

Adedoyin, O. B., and Soykan, E. (2020). Covid-19 Pandemic and Online Learning: the Challenges and Opportunities. Interactive Learn. Environments, 1–13. doi:10.1080/10494820.2020.1813180

CrossRef Full Text | Google Scholar

Beekman, J. (2021). Positive and Negative Aspects of Remote Learning. Austin, Texas: IoT Marketing. Available at https://iotmktg.com/positive-and-negative-aspects-of-remote-learning/.

Google Scholar

Byers, K. M., Elsayed-Ali, S., Jarjue, E., Kamikubo, R., Lee, K., Wood, R., et al. (2021). “Reflections on Remote Learning and Teaching of Inclusive Design in HCI,” in 3rd Annual Symposium on HCI Education at CHI, (EduCHI’21).

Google Scholar

Churchill, E. F., Bowser, A., and Preece, J. (2013). Teaching and Learning Human-Computer Interaction. Interactions 20, 44–53. doi:10.1145/2427076.2427086

CrossRef Full Text | Google Scholar

Educause Review (2020). The Difference between Emergency Remote Teaching and Online Learning. Available at https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning (Accessed October 1, 2021).

Google Scholar

Hodges, C., Moore, S., Lockee, B., Trust, T., and Bond, A. (2020). The Difference between Emergency Remote Teaching and Online Learning. Educ. Rev. Available at: https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-andonline-learning (Accessed October 1, 2021).

Google Scholar

ispring (2021). A Learning Management System (LMS): Everything There Is to Know. Available at https://www.ispringsolutions.com/blog/what-is-lms? (Accessed November 18, 2021).

Google Scholar

Langford, M., and Damsa, C. (2020). Online Teaching in the Time of COVID-19: Academics’ Experience in Norway. Oslo, Sweden: Centre for Experiential Legal Learning (CELL), part of the University of Oslo. Available at https://www.jus.uio.no/cell/ (Accessed November18, 2021).

Google Scholar

O’Malley, B. (2020). Leaders Say, ‘We Were Not Ready for COVID-19’. London, United Kingdom: University World News. Available at https://cutt.ly/xjdT3y3 (Accessed November 23, 2021).

Google Scholar

Oleson, A. (2020). Seven Tips to Improve HCI Education. Bits and Behavior. Available at https://medium.com/bits-and-behavior/seven-tips-to-improve-hci-education-fa64db5da4f5 (Accessed October 1, 2021).

Google Scholar

Schlesselman, L. S. (2020). Perspective from a Teaching and Learning Center during Emergency Remote Teaching. Ajpe 84, ajpe8142. doi:10.5688/ajpe8142

PubMed Abstract | CrossRef Full Text | Google Scholar

Sharp, H., Preece, J., and Rogers, Y. (2019). Interaction Design ‐ beyond Human‐Computer Interaction. 5th Ed. West Sussex, United Kingdom: John Wiley & Sons.

Google Scholar

Souleles, N., Laghos, A., and Savva, S. (2021). From Face-To-Face to Online: Assessing the Effectiveness of the Rapid Transition of Higher Education Due to the Coronavirus Outbreak–The Student Perspective. Proc. INTED2021 Conf. 8 (9th)., p.

Google Scholar

Stirling Institute of Australia (2020). Benefits of Practical Learning. Available at https://sia.edu.au/students-corner/benefits-of-practical-learning (Accessed September 22, 2021).

Google Scholar

Thurab-Nkhosi, D., Maharaj, C., and Ramadhar, V. (2021). The Impact of Emergency Remote Teaching on a Blended Engineering Course: Perspectives and Implications for the Future. SN Soc. Sci. 1, 159. doi:10.1007/s43545-012-00172-z10.1007/s43545-021-00172-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Tullis, T., and Albert, B. (2013). Measuring the User Experience – Collecting, Analyzing, and Presenting Usability Metrics. 2nd Ed. Amsterdam: Elsevier/Morgan Kaufmann.

Google Scholar

Turnitin (2021). Turnitin: Empower Students to Do Their Best, Original Work. Available at https://www.turnitin.com (Accessed November 18, 2021).

Google Scholar