Henning Pohl
Aske Mottelson- 1Department of Computer Science, Aalborg University, Aalborg, Denmark
- 2Department of Digital Design, IT University of Copenhagen, Copenhagen, Denmark
We created a hand texture resource (with different skin tone versions as well as non-human hands) for use in virtual reality studies. This makes it easier to run lab and remote studies where the hand representation is matched to the participants’ own skin tone. We validate that the virtual hands with our textures align with participants’ view of their own real hands and allow to create VR applications where participants have an increased sense of body ownership. These properties are critical for a range of VR studies, such as of immersion.
1 Introduction
We release Hafnia Hands1, a set of hand textures for use in virtual reality (VR) research. The textures cover all six skin-tone levels of the Fitzpatrick scale (Fitzpatrick, 1988) in addition to three non-human variants.
Hand representations are an increasingly common way for users to interact in virtual environments and the hand appearance an important part of the user experience and bodily self-consciousness when immersed. Our resource allows researchers to easily setup VR studies with visually realistic hand representations based on a standard skin tone scale.
Virtual hand representations are increasingly enabled by consumer-oriented VR technology through built-in hand tracking support (e.g., the Oculus Quest2) as well as add-on products that can add hand tracking support to headsets that do not otherwise support it (e.g., the Leap Motion3). While these systems usually provide hand models that work with their tracking, texturing of these models is limited. For example, the Oculus and Leap Motion SDKs both do not include hand textures.
Unfortunately, creating detailed textures for existing hand models requires the expertise of a texture artist—a role not typically found in research teams. There are textured hands available for purchase, but these usually use different underlying 3d models and hence are not directly compatible with the hand tracking data. While it can be possible to adapt or create a hand skeleton that matches a model to that data, that in itself is a task that commonly requires expertise from a 3d artist. For researchers that want to work with hand representations, but do not have expertise in 3d modeling, rigging, and texturing, there hence is a lack of easy to use options.
1.1 Applications for Textured Hands in VR
There are a range of VR research topics where hand textures are useful. In particular, we see applications for the provided resource in 1) embodiment research and 2) conducting remote VR studies.
1.1.1 Embodiment Research
In VR, participants can experience illusory ownership of any imaginable body, making it an increasingly important research tool for understanding fundamental questions about the relationship between the body and the mind (Slater et al., 2009; Maister et al., 2015).
Embodiment has been referred to participants feeling that, when given a virtual body, that body’s “properties are processed as if they were the properties of one’s own biological body” (Kilteni et al., 2012). One component of that is body ownership, the notion that a participant’s “body is the source of the experienced sensations” (Kilteni et al., 2012). The relationship between the virtual body and the sense of body ownership is complex (Maselli and Slater, 2013).
Avatars, including hands, play an important role in embodiment research. For example, more human-looking avatars do not necessarily lead to more body ownership (Lugrin et al., 2015). The sense of body ownership differs not just by visual realism, but also depending on other properties, such as perspective (Maselli and Slater, 2013), or whether the hands are connected (Seinfeld and Müller, 2020). Furthermore, VR embodiment research lacks standardized measures, tasks, or procedures, making it even more challenging to converge on theory-building.
The hand textures provided, can therefore aid three common challenges in VR embodiment research: 1) to match participants’ real skin-tone to not conflict embodiment measures; 2) to enable research in the important of visual congruence (e.g., by mismatching participants’ skin tone and virtual hand texture); and 3) to create a new standard procedure for employing humans hands in VR embodiment research.
1.1.2 Remote VR Studies
The infrastructure for remote VR studies is still developing and an active research area, and with the recent growing adoption of consumer-level VR devices, remote VR studies have now become a feasible alternative to lab studies (Mottelson et al., 2021; Ratcliffe et al., 2021). Such crowdsourced VR experiments can replicate lab studies (Ma et al., 2018) and enable more seamless recruitment of larger numbers and more diverse participants. Furthermore, remote studies can happen even at times when access to laboratories is restricted (Steed et al., 2020).
Before consumer VR devices were commonly available, previous studies have distributed Google Cardboard kits to participants (Mottelson and Hornbæ k, 2017; Steed et al., 2016). Similarly, researchers have shown the feasibility of tapping into the exiting VR software ecosystem, for example, running studies inside the VRChat application (Saffo et al., 2020).
Currently, the most popular VR devices for consumers are the Oculus Quest one and 2. Both come with hand tracking support and thus allow for running a range of studies that benefit from this level of fidelity (e.g., studies of grasping, embodiment, or gesturing). As remote studies commonly have a diverse set of participants, it is important that the hand representation they work with reflects this diversity. Hence, the skin-tone variants we make available are an important component required for running these studies.
1.2 VR Assets as Research Artifacts
With our hand texture resource, we provide a piece of infrastructure for VR researchers. While VR studies require a substantial amount of technical expertise to set up, resources like ours can ease that burden, but also allow for more comparable results across studies. In recent years, there has been a growing number of scientific resources similar to the Hafnia Hands. For example, Regal et al. (2018) made assets available that make it easier to integrate questionnaires into VR scenes. With NavWell, Commins et al. (2020) released a tool for creating and running navigation experiments. This standardizes the study format, but also substantially lowers the barriers for running such a study. Most closely related to our hand texture resource is the Microsoft Rocketbox avatar library (Gonzalez-Franco et al., 2020), containing many rigged humanoid 3D models readily available as self-avatars, or as agents within VR environments.
2 Methods
We commissioned the set of nine hand texture variants from a 3d artist on Fiverr. Fiverr is a platform for connecting with freelancers, focused on creative tasks such as video editing or 3d modeling. The set (shown in Figure 1) includes six different human skin-tones, as well as three non-human hands: a robot hand, an alien hand, and a skeleton hand. Each variant comes with albedo, ambient occlusion, metallic, smoothness/roughness, and normal maps and hence is compatible with commonly used physically-based shading models (such as the Unity Standard Shader).
FIGURE 1. We provide nine different hand textures: six skin-tone variants, an alien, a robot, and a skeleton hand. The skin-tone variants are designed to look realistic, while the non-human hands are more stylized.
The textures are designed to work directly with the default hand used by the Oculus Quest hand tracking. Currently, this is the most widely used variant of hands for VR and also the one most suited for remote studies due to the large user base of consumers. However, note that the textures can, in principle, be used with other hand models. Assuming the topology is similar (as should be the case for models of human hands) this would only require some changes to the UV coordinates of those models. Additionally, we provide hand textures compatible with the 3D hand models developed for the Oculus controllers; that way researchers can utilize our hand resource for studies using both free hand interaction and controller-based interaction.
We conducted an online and unsupervised study to evaluate the suitability of the hands for remote evaluations, and to get quantitative insights of their effects on embodiment. We evaluate the effects of skin-tone matching as well as owning the non-human hand variants. The study, its hypotheses, method, procedure, and analyses approach was pre-registered4.
We used a within-subjects design with hand texture as the only independent variable. Hand texture had three levels: skin-tone matched, skin-tone mismatched, and non-human. Participants completed three repetitions of a hand movement task with each hand texture, balanced using a latin square, for a total of nine trials per participant.
2.1 Participants
We recruited 112 participants from an internal email list of previous participants. Participants were predominantly male (103 male, 6 female, and 3 participants who did not want disclose gender) and young, with an estimated mean age of 28.9 (SD 9.6). The median reported skin tone was 3. Most participants reported medium skin tones, with 86% in the 2–4 range on the Fitzpatrick scale (6 × 1, 36 × 2, 36 × 3, 24 × 4, 7 × 5, and 3 × 6 on the Fitzpatrick scale). Participants conducted the study from 32 different countries, most commonly the USA (18), Poland (13), Germany (12), the UK (12), and Italy (8). Most participants were also experienced VR users, with 51 having more than 100 h of VR experience (as well as 36× 50–99 h, 23× 10–49 h, and 2× 5–9 h).
2.2 Task
We employed a novel study task where participants followed targets with their hands. The task required participants to act with their hands, while also forcing them to focus on their hands. Each trial took 60 s and was split in two equally long phases: one for each hand, ordered randomly.
During the task, participants had to keep their hand inside a framed box that appeared in front of them. After appearing, the box moved along a randomly generated path within a 40, ×, 60, ×, 40 cm volume. We generated the path so each segment was at least 20 cm long. Box movement slowed down at path nodes and accelerated in between, moving at an average overall speed of 0.6 m per second. During each segment the box also performed a random rotation to between − 40°and 90°(left hand) − 90°and 40°(right hand) roll. The boxes changed color to indicate whether participants’ hand were contained within, prompting participants to follow study procedures. When participants moved outside the boxes, we displayed a textual alert in the background asking them to return their hand inside.
2.3 Procedure
We first prompted participants for informed consent and collected demographic information, using a virtual questionnaire. We then asked participants about their skin tone, which then formed the selection for the skin-tone matched condition. For the skin-tone mismatched condition, we picked a hand texture three levels away from the participant’s (i.e., 1 → 4, 3 → 6, 5 → 2, … ). We always used the alien hand texture for the non-human condition. Participants then completed nine trials (three with each hand) in random order. Overall, the study took about 10 min. A first warm-up round was used to determine tracking quality. If mean confidence for tracking was below 60%, the participant would retry until quality was satisfactory, as per (Mottelson et al., 2021).
2.4 Measures
For each trial we measured four dependent subjective variables using questionnaires displayed in VR: To measure Resemblance we used the Features question from (Banakou and Slater, 2014). To measure Agency we used the Agency question from (Banakou and Slater, 2014). To measure Body Ownership we used the MyBody question from (Banakou and Slater, 2014). To measure Humanness we used a scale from (Ho and MacDorman, 2017), composed of five questions. Please see Table 1 for details.
TABLE 1. Measures used during validation.
3 Validation
From the participants responses we seek to validate the suitability of Hafnia hands for VR research. In particular, when used for embodiment and remote studies, Hafnia hands need to 1) resemble participants’ own hands and hence also elicit a higher sense of body ownership. See Figure 2 for an overview of how participants’ rated their experience with the hands.
FIGURE 2. Participants’ responses to the four subjective measures we used for validating our hand texture dateset. Responses range from -3 (strongly disagree) to 3 (strongly agree), humanness shows aggregated scale based on five such responses.
As an initial test, we checked whether there were differences between the responses of male and female participants. Fisher’s exact tests show no significant differences between the sexes (Body Ownership: p = 0.5, Agency: p = 0.5, Resemblance: p = 0.7, and Humanness: p = 0.1).
We test the effects of hand representation on body ownership, agency, resemblance, and humanness using Friedman tests for main effects. We use Wilcoxon rank sum tests with Bonferroni correction for post-hoc comparisons.
We find an effect of hand representation on body ownership; χ2 (2) = 109.77, p < 0.001. All pairwise comparisons were significant (p < 0.001). For agency, there was a significant main effect χ2 (2) = 26.27, p < 0.001. Yet, only the alien hand and matched hand were significantly different (p = 0.04). We also found an effect of hand representation on resemblance; χ2 (2) = 150.31, p < 0.001. All pairwise comparisons were also significant (p < 0.001). Finally, we also found a significant effect of hand representation on humanness; χ2 (2) = 141.7, p < 0.001. Post-hoc tests found significant differences between the alien hand and the two realistic hands (p < 0.001).
In summary, the hands provided as part of Hafnia hands showed increased body ownership using a skin-tone matched texture; increased resemblance with a skin-tone matched texture; and decreased humanness with a alien texture. All registered hypotheses were hence found to be true.
These results also validate the suitability of Hafnia hands as a resource in embodiment and remote VR studies. Particularly with respect to body ownership, skin-tone matched hands provide a benefit. Furthermore, we validated that the provided non-human alien hand elicits the opposite response: low body ownership, visual resemblance, and humanness. Hence, the set of provided hand textures covers a wide spectrum of subjective responses, which is required in a range of study designs.
4 Limitations
The sample for the evaluation was predominantly young and male, which has previously been shown to cause bias in VR research (Peck et al., 2020). The hand textures visually resemble relatively young human hands, making the sample’s age fit the study; however, future studies could entail textures matched and evaluated for seniors. The hand textures were, however, designed as gender neutral, whereas the evaluation consisted of predominantly males. While we saw no significant differences between male and female responses, this still is a limitation of the evaluation, caused the bias of HMD owners’ demographics. Our participant sample also is biased in terms of skin tone with 96 of them of type II, III, or IV. We hence have only little data from people with very light or dark skin.
As a manually-created resource, Hafnia hands are limited in the range of visual characteristics they cover. In addition to lack of gender and age specificity, the hands also do not account for several other properties, such as: dirtiness, fingernail length/style/color, skin conditions, or finer nuances in skin-tone. A potential solution for this are parametric hand texture models (Qian et al., 2020). However, to make use of the wide range of such models, a parameter acquisition step for each participant is required. The quality of current model outputs also is not consistently at the same level as artist-created textures and does not include non-human textures at all.
While hands are perhaps the most important limb in embodiment research (e.g., the rubber hand illusion), future research resources and studies could consider creating resources enabling full body VR embodiment studies.
Data Availability Statement
The hand textures described in this article as well as the data from their evaluation can be found at: https://github.com/henningpohl/hafnia-hands.
Author Contributions
All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication. Both HP and AM designed, implemented, and conducted the evaluation, HP conducted the statistical analyses; HP made figures, and HP and AM wrote the manuscript.
Funding
This research was supported by UCPH’s Data+ pool under the agreement ‘Quantifying Body Ownership’.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Acknowledgments
We’d like to thank Ranjeet Singh5 for creating the hand textures.
Footnotes
1Named after the city of Copenhagen
2https://www.oculus.com/quest-2/
3https://www.ultraleap.com/product/vr-developer-mount/
5https://www.fiverr.com/google_jatt
References
Banakou, D., and Slater, M. (2014). Body ownership causes illusory self-attribution of speaking and influences subsequent real peaking. Proceedings of the National Academy of Sciences 111, 17678–17683. doi:10.1073/pnas.1414936111
Commins, S., Duffin, J., Chaves, K., Leahy, D., Corcoran, K., Caffrey, M., et al. (2020). Navwell: A simplified virtual-reality platform for spatial navigation and memory experiments. Behavior Research Methods 52, 1189–1207. doi:10.3758/s13428-019-01310-5
Fitzpatrick, T. B. (1988). The Validity and Practicality of Sun-Reactive Skin Types I Through VI. Archives of Dermatology 124, 869–871. doi:10.1001/archderm.1988.0167006001500810.1001/archderm.124.6.869
Gonzalez-Franco, M., Ofek, E., Pan, Y., Antley, A., Steed, A., Spanlang, B., et al. (2020). The rocketbox library and the utility of freely available rigged avatars. Frontiers in Virtual Reality 1, 20. doi:10.3389/frvir.2020.561558
Ho, C.-C., and MacDorman, K. F. (2017). Measuring the uncanny valley effect. International Journal of Social Robotics 9, 129–139. doi:10.1007/s12369-016-0380-9
Kilteni, K., Groten, R., and Slater, M. (2012). The sense of embodiment in virtual reality. Presence: Teleoperators and Virtual Environments 21, 373–387. doi:10.1162/PRES_a_00124
Lugrin, J., Latt, J., and Latoschik, M. E. (2015). “Avatar anthropomorphism and illusion of body ownership in vr,” in 2015 IEEE Virtual Reality (VR), Arles, France, 23-27 March 2015, 229–230. doi:10.1109/VR.2015.7223379
Ma, X., Cackett, M., Park, L., Chien, E., and Naaman, M. (2018). Web-based vr experiments powered by the crowd. Republic and Canton of Geneva, CHE: International World Wide Web Conferences Steering Committee, 33–43. WWW ’18. doi:10.1145/3178876.3186034
Maister, L., Slater, M., Sanchez-Vives, M. V., and Tsakiris, M. (2015). Changing bodies changes minds: owning another body affects social cognition. Trends in Cognitive Sciences 19, 6–12. doi:10.1016/j.tics.2014.11.001
Maselli, A., and Slater, M. (2013). The building blocks of the full body ownership illusion. Frontiers in Human Neuroscience 7, 83. doi:10.3389/fnhum.2013.00083
Mottelson, A., and Hornbæk, K. (2017). “Virtual reality studies outside the laboratory,” in Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology (New York, NY, USA: Association for Computing Machinery). VRST ’17. doi:10.1145/3139131.3139141
Mottelson, A., Petersen, G. B., Lilija, K., and Makransky, G. (2021). Conducting unsupervised virtual reality user studies online. Frontiers in Virtual Reality 2:681482. doi:10.3389/frvir.2021.681482
Peck, T. C., Sockol, L. E., and Hancock, S. M. (2020). Mind the gap: The underrepresentation of female participants and authors in virtual reality research. IEEE Transactions on Visualization and Computer Graphics 26, 1945–1954. doi:10.1109/TVCG.2020.2973498
Qian, N., Wang, J., Mueller, F., Bernard, F., Golyanik, V., and Theobalt, C. (2020). “HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization,” in Proceedings of the European Conference on Computer Vision (ECCV) (Springer). doi:10.1007/978-3-030-58621-8_4
Ratcliffe, J., Soave, F., Bryan-Kinns, N., Tokarchuk, L., and Farkhatdinov, I. (2021). “Extended reality (xr) remote research: A survey of drawbacks and opportunities,” in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery). CHI ’21. doi:10.1145/3411764.3445170
Regal, G., Schatz, R., Schrammel, J., and Suette, S. (2018). “Vrate: A unity3d asset for integrating subjective assessment questionnaires in virtual environments,” in 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX), 1–3. doi:10.1109/QoMEX.2018.8463296
Saffo, D., Yildirim, C., Di Bartolomeo, S., and Dunne, C. (2020). Crowdsourcing virtual reality experiments using vrchat. New York, NY, USA: Association for Computing Machinery, 1–8. CHI EA ’20. doi:10.1145/3334480.3382829
Seinfeld, S., and Müller, J. (2020). Impact of visuomotor feedback on the embodiment of virtual hands detached from the body. Scientific Reports 10, 22427. doi:10.1038/s41598-020-79255-5
Slater, M., Pérez Marcos, D., Ehrsson, H., and Sanchez-Vives, M. (2009). Inducing illusory ownership of a virtual body. Frontiers in Neuroscience 3. Article 29. doi:10.3389/neuro.01.029.2009
Steed, A., Frlston, S., Lopez, M. M., Drummond, J., Pan, Y., and Swapp, D. (2016). An ‘in the wild’ experiment on presence and embodiment using consumer virtual reality equipment. IEEE Transactions on Visualization and Computer Graphics 22, 1406–1414. doi:10.1109/TVCG.2016.2518135
Keywords: skin tone, virtual reality, hand tracking, avatars, presence
Citation: Pohl H and Mottelson A (2022) Hafnia Hands: A Multi-Skin Hand Texture Resource for Virtual Reality Research. Front. Virtual Real. 3:719506. doi: 10.3389/frvir.2022.719506
Received: 02 June 2021; Accepted: 13 April 2022;
Published: 26 May 2022.
Edited by:
Catherine Sotirakou, University of Vienna, AustriaReviewed by:
Miguel A Otaduy, Rey Juan Carlos University, SpainMarkus Kemmelmeier, University of Nevada, Reno, United States
Copyright © 2022 Pohl and Mottelson. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Henning Pohl, aGVubmluZ0Bjcy5hYXUuZGs=