Investigating body perception of multiple virtual hands in synchronized and asynchronized conditions

As part of research on human augmentation, multiple bodies are used in a virtual environment. For example, a study on multiple partial body parts has been conducted using up to 64 hands and showed that multiple hands reduced the distance traveled by one hand. However, body perception has yet to be verified. In this study, we investigated how body perception changes when nine hands, partial bodies, are moved synchronously in a virtual environment, compared to a single hand. In addition, we examined whether the sense of body ownership for all nine hands was elicited simultaneously or whether it was elicited for some of the hands while switching between them. Participants performed a reaching task using one or nine hands presented in a virtual environment. After the reaching task, a threat stimulus was given, and hand movements in response to the threat were measured. After completion of each condition, the subjective sense of body ownership and sense of agency was investigated using a Likert scale. The results indicated that users felt the sense of body ownership of several hands for the nine hands and manipulated them by switching their attention to multiple bodies.

1 Introduction

1.1 Sense of body ownership and sense of agency

Body perception, which is the awareness of one’s body, includes the sense of body ownership and the sense of agency. The sense of body ownership is the feeling that the body, or a part of the body, belongs to oneself (Gallagher, 2000). The sense of agency is that the action was caused by oneself (Gallagher, 2000). The sense of body ownership is induced by visual information and haptic or motor synchronization, available even in the non-native body. One of the most representative studies on body ownership is the rubber hand illusion, in which visual-tactile synchronization of visual information and tactile stimuli was used (Botvinick and Cohen, 1998). In this illusion, which conceals the participant’s real hand, the participant observes a rubber hand placed in a natural-looking position, and when the real hand and the rubber hand are simultaneously given similar tactile stimuli, the participant feels the rubber hand as if it were his or her body. The effect of the rubber hand illusion was reported to be more potent when attention was directed to the rubber hand rather than the real hand (Jenkinson et al., 2013). In a study using visual-motor synchronization, it was reported that the illusion of the sense of body ownership was also induced by observing a mannequin’s hand synchronized with the subject’s hand movements (Kalckert and Ehrsson, 2012; Kalckert and Ehrsson, 2014). Such an illusion of body ownership is also induced in virtual bodies. In the virtual hand illusion, the virtual hand was perceived as if it were one’s hand by visual-tactile synchronization between the virtual hand and the participant’s hand (Slater et al., 2008), or by visual-motor synchronization between the virtual hand and the participant’s hand (Sanchez-Vives et al., 2010). The sense of body ownership of these virtual hand illusions was examined by comparing them with asynchronous tactile stimuli and motion, which do not elicit the sense of body ownership. A sense of body ownership was also elicited for tools that were functionally similar to real hands (Cardinali et al., 2021). On the other hand, it was reported that a realistic hand shape has a higher sense of body ownership than a controller or abstract hand (Argelaguet et al., 2016; Lougiakis et al., 2020; Venkatakrishnan et al., 2023). The sense of body ownership was also examined in experiments using various conditions, such as tactile feedback (Jahanian Najafabadi et al., 2023a; Jahanian Najafabadi et al., 2023b), sound feedback (Canales and Jörg, 2020), finger tracking (Adkins et al., 2021), and changes in the way objects were grasped (Canales et al., 2019). It was also reported that training improved the sense of body ownership (Schone et al., 2024). In this experiment, a glove similar to a real hand was used to examine the sense of body ownership under synchronous and asynchronous conditions.

1.2 Visual-tactile synchronization and visual-motor synchronization

The difference between visual-tactile and visual-motor synchronization is that only visual-motor synchronization elicits the sense of agency (Gallagher, 2000), the feeling that one has caused the movement. The sense of agency, the awareness of one’s body, is integral to body perception. In addition, Kilteni et al. (2012a) proposed that the three components of the Sense of Embodiment are the sense of body ownership, the sense of agency, and the sense of self-location, suggesting the importance of the sense of agency in body perception. This sense of agency is different from the sense of body ownership. The sense of agency was reported to be felt even under conditions that do not induce the sense of body ownership, such as a 180-degree rotation of the model hand, which is anatomically impossible (Kalckert and Ehrsson, 2012). In a study comparing the effects of agency on the sense of body ownership, the illusion of agency in visual-tactile synchronization, active visual-motor synchronization, and passive visual-motor synchronization was compared using the rubber band illusion, and the results showed that the illusion was equally strong in all three cases (Kalckert and Ehrsson, 2014). The authors considered that there was no difference in the sense of body ownership between visual-tactile synchronization and visual-motor synchronization because visual-motor synchronization provides more information but also provides more information that causes a discrepancy between the senses. Thus, while the sense of agency does not affect the strength of the sense of body ownership in visual-motor synchronization and visual-tactile synchronization, the importance of the sense of agency as a component of the body sense is indicated. Visual-motor synchronization allows the body to move freely and is expected to be applied to the real world. Therefore, the visual-motor synchronization method was used in this study.

1.3 Body ownership and startle response

The startle response to a threat stimulus is sometimes used as a method to verify the body ownership. In the rubber hand illusion, it was reported that the threat response to a rubber hand is associated with the assimilation of the rubber hand to one’s body image by evaluating the skin conductance response (SCR) (Armel and Ramachandran, 2003). In this case, the objective measure of SCR showed that participants perceived strong SCR when the rubber hand, which generated the sense of body ownership, was injured. In studies using indices of other objective measures, various rating scales were used for startle response, including heart rate (Slater et al., 2010), brain response (Ehrsson et al., 2007), physical avoidance behavior toward the threat (Gonzalez-Franco et al., 2010), and changes in Electromyography (EMG) associated with avoidance (Tsuji et al., 2013). These threat stimuli have also been introduced to virtual bodies in virtual environments; there were studies in which the objects themselves induced threats, such as a knife stabbing a virtual body (González-Franco et al., 2014; Zhang et al., 2015), a saw slashing a virtual body (Kilteni et al., 2012b), the appearance of a spinning saw (Argelaguet et al., 2016; Canales et al., 2019; Canales and Jörg, 2020), the appearance of a spiky ball (Canales et al., 2019; Canales and Jörg, 2020), or the presentation of an obstacle during a task (Lougiakis et al., 2020). Some studies utilized changes in the user’s body motion caused by threat stimulus in the virtual environment (Fribourg et al., 2018) and studies that use virtual environments such as lamps falling (Yuan and Steed, 2010) or fans descending (Gonzalez-Franco et al., 2010) in the virtual environment. In this study, we quantified and analyzed the aspect of the startle response, namely, jerk (Flash and Hogan, 1985; Hagiwara et al., 2020), to examine the rapid change in acceleration responses to instantaneous threat stimulus that focus attention on multiple hands. Jerk is a physical quantity that indicates the time derivative of acceleration and is commonly used to evaluate smooth body motion and embodiment based on optimizing control to minimize abrupt changes in motion. This study used it to measure the involuntary movement associated with physical avoidance. We also designed an experiment using knives that enabled subjects to induce localized threats by varying the position and number of knives given to them.

1.4 Multiple bodies toward body perception

A study of body perception toward multiple bodies reported that when an illusory sense of body ownership was generated for a rubber hand, the sense of body ownership was simultaneously generated for the real hand (Guterstam et al., 2011). Research has also been on the illusion of the sense of body ownership for multiple virtual bodies using virtual environments. It was shown that when the two virtual bodies were observed from behind, the subjects’ backs and backs were simultaneously stimulated, providing the sense of body ownership over the two virtual bodies (Heydrich et al., 2013). However, the third-person viewpoint produced a weaker sense of body ownership than the first-person viewpoint, indicating that the first-person viewpoint is essential for body perception toward virtual bodies (Petkova et al., 2011; Maselli and Slater, 2013). In response, Guterstam et al. (2020) presented two bodies lying next to each other from the first-person perspective and showed that visual-tactile synchronization of the stimuli produced the sense of possession of the two whole bodies. In these studies, combining visual-tactile stimuli induced the sense of body ownership over multiple bodies but not manipulation of those bodies. It is also presumed that the whole body occupies a large visual field area, making it difficult to handle more of the whole body from a first-person perspective. In contrast, Miura et al. (2022) showed that by dividing the viewpoints of up to four virtual bodies into a single screen and displaying them on a single screen and by having the four virtual bodies move in synchronization with the observer’s movements, the observer feels each virtual body as if it were his or her body. On the other hand, it should be considered that each of the four virtual body perspectives was presented in parallel on a single screen and, therefore, not a complete first-person perspective; in addition, no comparison was conducted using the synchronous and asynchronous conditions of the movements. Their study also reported that the four bodies were manipulated by switching attention to sensory information between the bodies while feeling the sense of body ownership over the four bodies, presenting the possibility that switching is necessary to perform intentional behaviors (Miura et al., 2022). Therefore, when more bodies were presented from the first-person perspective, it needed to be clarified whether the sense of body ownership was generated in all bodies simultaneously or whether the sense of body ownership was generated for some bodies while switching attention among multiple bodies. Although these studies were on whole bodies, studies have been conducted to increase the number of partial bodies to multiple bodies. Schjerlund et al. (2021) experimented with handling up to 64 hands by arranging multiple bodies in a cube shape and showed that multiple hands reduce the distance traveled by real hand. In this study, more bodies were handled by using partial-body hands. However, body perception, including the sense of body ownership, has yet to be reported. Therefore, we investigated body perception when more partial bodies are manipulated by visual-motor synchronization in the first-person perspective, using synchronous and asynchronous movement conditions.

1.5 Aim

We examined whether visual-motor synchronization from a first-person perspective allows the user to gain the sense of body ownership and the sense of agency as multiple bodies when nine hands are presented, compared to a single synchronized hand or nine asynchronized hands. We also examined whether the sense of body ownership is generated at the same time or by switching attention to the nine hands, which have a more significant number of bodies. In the experiment, participants performed a reaching task with one or nine virtual hands whose movements are synchronized or asynchronized, followed by a threat stimulus (knife) event. Participants were equipped with a tracker that measured hand movement during the experiment. We investigated whether the jerk: the objective measure of startle response in this study changes when the target hand of the threat stimulus (presence or absence and number of attention) and the timing of the threat stimulus are changed. At the end of the experiment, participants were asked to answer a questionnaire to examine their body perceptions when multiple bodies were used.

2 Materials and methods

2.1 Task environment

Participants were seated, wearing VIVE PRO EYE, a virtual reality Head Mounted Display (HMD) (Figure 1A), and immersed in a virtual environment constructed in Unity. The virtual environment was a 10 m × 5 m x 10 m room, displaying a hand performing a reaching task and a white and red sphere. The size of the hands was the default hand size in the SteamVR framework, and the diameter of each sphere was 15 cm. Each sphere was always hidden and appeared only during the reaching task. A VIVE tracker was worn to track and measure the participant’s hand movements. The hand tracker was attached to the back of the right hand with a hand strap, allowing for hands-free movement throughout the experiment.

Figure 1

Figure 1. Example of participants in the experiment (A), the red sphere as the target for the reaching task (B), and the white sphere as the origin of the reaching task (C).

2.2 Visualization of nine hands

To examine multiple bodies’ perceptions of the hand, position and rotation data obtained from the hand tracker were synchronized with the movements of nine virtual hands. Virtual hands were added to the room and aligned at 0.5 m intervals in a 3 × 3 dotted grid pattern so that all hands appeared uniform (Figure 2A). In addition, the hand at the most front of the middle row of nine hands was in the same position as the participant’s physical hand. In this way, participants were able to synchronously manipulate all hands from a first-person perspective as if they were their own hands. The nine hands were arranged so that all hands were observed from an overhead view, while only some were observed when the viewpoint was changed during the task.

Figure 2

Figure 2. Hands appearing in the experiment. Nine hands arrangement (A), one hand (B). Red dotted circle represents the hand aligned with the physical hand.

2.3 Participants

Eighteen males and two females (M = 23.55 years, SD = 1.877) participated in Experiment 1, and eighteen males and two females (M = 23.50 years, SD = 1.878) participated in Experiment 2. Both experiments were within-subjects designs. Nineteen of the twenty participants in Experiment 1, except one, participated in Experiment 2. They had healthy vision and physical abilities and were all right-handed. To prevent learning effects from occurring if they participated in Experiments 1 and 2, they were separated by at least 1 day. Written informed consent was given before the experiments. This study was conducted using the experimental protocol approved by the Research Ethics Committee of the Faculty of Science and Technology, Keio University.

2.4 Task and threat stimulus

The white sphere was the origin, and the red sphere was the target of the reaching task (Figures 1B, C). In order to ensure that movement during the task did not affect the measurement of the jerk against the threat stimulus, each ball disappeared after touching for 2 s. In the trial, participants reached from the origin to the target position and returned to the origin using a virtual hand. The target red sphere had to be touched 27 times to complete one condition. The red ball was placed within 0.2 m of each hand. In the case of multiple hands, a red sphere was presented once near each of the nine hands. The order of the red spheres was randomized, and this was repeated three times, resulting in a total of 27 red spheres being presented. In the case of nine virtual hands, the position of the center-nearest hand had the exact coordinates of the physical hand. Thus, the white ball had to be reached by that hand. Immediately after completing the reaching task, a knife was stabbed into the hand as the threat stimulus. The knife was stabbed under different conditions in each experimental condition, and we investigated how this generated the sense of body ownership.

2.5 Measurement of startle response

We obtained the position of a tracker attached to the hand to measure the jerk, which indicates the hand’s motion in response to a threat stimulus. The jerk was acquired as the third derivative of the tracker’s position. It is generally reported that responding to a visual stimulus takes about 0.2 s (Woods et al., 2015). However, this reaction time differs among individuals and situations. Furthermore, in this study, we considered detecting the most intense reactive movement of the hand. Therefore, we used the maximum value of the jerk up to 1 s after the onset of the threat stimulus.

2.6 Questionnaire

A questionnaire was administered at the end of each experimental condition. The questionnaire for multiple bodies was custom developed with reference to the study of Miura et al. (2022) because it did not accommodate the standardized questionnaire (Peck and Gonzalez-Franco, 2021). The questionnaire included three questions on body ownership, three on the sense of agency, two on multiple hands perception, three on threat stimulus, and three on switching between multiple hands perceptions, for 14 questions (Table 1). The questionnaire asked participants to rate their perceptions on a 7-point Likert scale ranging from −3 (strongly disagree) to 3 (strongly agree).

Table 1

Table 1. The scripts of questionnaires.

2.7 Experiment 1

We investigated the body perception for nine hands compared to one hand (Figures 2A, B). There were four physical conditions: nine synchronized hands (9 Sync), nine asynchronous hands (9 Async), one synchronized hand (1 Sync), and one asynchronous hand (1 Async). In the asynchronous condition, the participants were presented with a 2-s delay from the real hand movement. Trials in each condition followed a Task and threat stimulus procedure. In all conditions, nine knives were presented as the threat stimulus; in the one hand condition, nine knives were presented in the exact location in the nine hands condition to provide the same stimulus as in the nine hands condition (Figures 3B, C). Before experimenting, participants were given practice trials in each physical condition. The practice trials were based on the Latin square, and the participants performed the four conditions one time, each according to the counterbalance. The practice trials were based on the Latin square, and the participants performed the four conditions according to the counterbalance. Each condition was repeated three times for a total of 12 jerk measurements. The order of the experiment was the random order of the four conditions, which were repeated three times. After the third repetition of each condition, participants were asked to respond to a questionnaire for that condition. The duration of the experiment was about 1.5 h.

Figure 3

Figure 3. Three threat stimuli were used to examine the startle response: a knife in one of the nine hands (A), nine knives in each of the nine hands (B), and nine knives appearing in one hand (C).

2.8 Experiment 2

We investigated the importance of attention in multiple bodies by comparing the threat stimulus to the one hand to which attention is directed with the threat stimulus to the one hand to which attention is not directed when nine hands were presented (Figure 2A). There were two physical conditions: nine synchronized hands and nine asynchronous hands. In the asynchronous condition, as in Experiment 1, participants were presented with a 2-s delay from the real hand movements. Trials in each condition followed the Task and threat stimulus procedure. There were also two threat stimulus conditions: one in which a knife was given to the hand that had performed the reaching task at the end of the task, i.e., the hand that was considered attentive, and one in which a knife was given to the hand that had not performed the reaching task at the end of the task, i.e., the hand that was considered inattentive (Figure 3A). Therefore, there were four experimental conditions: Attention Sync, No Attention Sync, Attention Async, and No Attention Async. In the threat stimulus to the hand not used for the task at the end of the task condition, the knife was inserted into a random hand among the last untasked hands. Before the experiment, participants were given practice trials in two physical conditions. The practice trials were based on the Latin square, and the participants performed the two conditions one time, each according to the counterbalance. The threat stimulus condition was varied in the experiment for each body condition. As in Experiment 1, the experiment was repeated three times for a total of 12 measurements. The order of the experiments was the random order of the four conditions, which were repeated three times. After the third repetition of each condition, participants were asked to respond to a questionnaire for that condition. The duration of the experiment was about 1.5 h.

3 Results

3.1 Experiment 1

The experiment was repeated three times with 20 participants. Thus, 60 jerk results were measured per condition. Since the questionnaire was administered only on the third trial, there were 20 data sets. We conducted the Shapiro-Wilk test to check the normality of the jerk results, and it was found that the data was not assumed to have been obtained from a normally distributed population. Therefore, the Wilcoxon signed rank test was performed to analyze the questionnaire and the jerk results. p-values were corrected by the Bonferroni correction.

3.1.1 Questionnaire

Figure 4 showed the results of the questionnaire for Experiment 1.

Figure 4

Figure 4. Results of the questionnaire for all participants in Experiment 1.

We summarized Q1 through Q3 regarding the sense of body ownership. In Q1, the participants were more likely to feel the sense of ownership as only one hand in the 1 Sync condition than in the 9 Sync (p < 0.001), 9 Async (p = 0.0014), and 1 Async (p = 0.033) conditions. They also felt the sense of ownership for one hand in the 1 Async condition than the 9 Sync (p = 0.0024) and 9 Async (p = 0.046) conditions. In Q2, they were more likely to feel the sense of body ownership toward the two to eight hands in the 9 Sync condition than in the 1 Sync (p = 0.0011), 9 Async (p = 0.040), and 1 Async (p = 0.0012) conditions. They also felt the sense of ownership for two to eight hands in the 9 Async condition than in the 1 Sync (p = 0.015) and 1 Async (p = 0.011) conditions. In Q3, they were more likely to feel the sense of body ownership toward the nine hands in the 9 Sync condition than in the 1 Sync (p = 0.0012) and 1 Async (p = 0.00275) conditions. They also felt the sense of ownership for the nine hands in the 9 Async condition than in the 1 Sync (p = 0.013) and 1 Async (p = 0.022) condition.

We summarized Q4 through Q6 regarding the sense of agency. In Q4, the participants were more likely to feel the sense of agency for only one hand in the 1 Sync condition than in the 9 Sync (p < 0.001) and 9 Async (p = 0.0015) conditions. They also felt the sense of agency for only one hand in the 1 Async condition, more than in the 9 Sync (p = 0.0026) and 9 Async (p = 0.071) conditions. In Q5, they were more likely to feel the sense of agency for two to eight hands in the 9 Sync condition than in the 1 Sync (p = 0.0018) and 1 Async (p = 0.0018) conditions. They also felt the sense of agency for two to eight hands in the 9 Async condition more than in the 1 Sync (p = 0.0041) and 1 Async (p = 0.0040) conditions. In Q6, they were more likely to feel the sense of agency for nine hands in the 9 Sync condition than in the 1 Sync (p = 0.0011), 9 Async (p = 0.020), and 1 Async (p = 0.011) conditions. They also felt the sense of agency for the nine hands in the 9 Async condition than in the 1 Sync (p = 0.0068) and 1 Async (p = 0.0059) conditions.

We summarized Q7 and Q8 for multiple hands perception. In Q7, no differences were found in either condition, and participants did not perceive any of the conditions as one significant move. In Q8, they recognized multiple hands more strongly in the 9 Sync condition than in the 1 Sync (p < 0.001), 9 Async (p = 0.045), and 1 Async (p = 0.0012) conditions. They also recognized multiple hands more strongly in the 9 Async condition than in the 1 Sync condition (p = 0.042).

We summarized the results of Q9 through Q11 for threat stimulus. In Q9, participants were more likely to feel threatened by only one hand in the 1 Sync condition than in the 9 Sync (p = 0.010), 9 Async (p = 0.025), and 1 Async (p = 0.034) conditions. They were also more threatened by only one hand in the 1 Async condition than in the 9 Sync condition (p = 0.041). In Q10, they were more likely to feel threatened by two to eight hands in the 9 Sync condition than in the 1 Sync (p = 0.0057) and 1 Async (p = 0.0011) conditions. They were also more threatened by two to eight hands in the 9 Async condition than in the 1 Sync (p = 0.0066) and 1 Async (p = 0.039) conditions. In Q11, they were more likely to feel threatened by nine hands in the 9 Sync condition than in the 1 Sync (p = 0.0012), 9 Async (p = 0.050), and 1 Async (p = 0.0018) conditions. They were also more threatened by nine hands in the 9 Async condition than in the 1 Sync (p = 0.0033) and 1 Async (p = 0.0048) conditions.

We summarized the results of Q12 through Q14 regarding attention switching to the hands. In Q12, participants were more likely to feel the sense of body ownership switching for the 9 Sync condition than for the 1 Sync (p = 0.0018) and 1 Async (p = 0.0012) conditions. They also felt the sense of body ownership switching in the 9 Async condition more than in the 1 Sync (p = 0.0087) and 1 Async (p = 0.0034) conditions. On the other hand, the 9 Sync condition was not associated with a stronger sense of body ownership switching than the 9 Async condition (p = 0.28). In Q13, they were more likely to feel attention switching in the 9 Sync condition than in the 1 Sync (p = 0.0015) and 1 Async (p = 0.0012) conditions. They also felt attention switching in the 9 Async condition more than in the 1 Sync (p = 0.0024) and 1 Async (p = 0.0073) conditions. In Q14, they were more likely to feel the sense of agency switching in the 9 Sync condition than in the 1 Sync (p = 0.0039) and 1 Async (p < 0.001) conditions. They also felt the sense of agency switching in the 9 Async condition more than in the 1 Sync (p = 0.0087) and 1 Async (p = 0.0038) conditions.

3.1.2 Jerk for the startle response

Figure 5 showed the results of the jerk for the startle response in Experiment 1. Participants responded more intensely to the threat stimulus in the 9 Sync condition than in the 9 Async condition (p = 0.025). They also responded more intensely to the threat stimulus in the 1 Sync condition than in the 9 Aync (p = 0.011) and 1 Async (p = 0.019) conditions.

Figure 5

Figure 5. Results of the jerk for all participants in Experiment 1. Error bars represent 95% confidence intervals.

3.2 Experiment 2

The experiment was repeated three times with 20 participants. Thus, 60 jerk results were measured per condition. Since the questionnaire was administered only on the third trial, there were 20 data sets. We conducted the Shapiro-Wilk test to check the normality of the jerk results, and it was found that the data was not assumed to have been obtained from a normally distributed population. Therefore, the Wilcoxon signed rank test was performed to analyze the questionnaire and the jerk results. p-values were corrected by the Bonferroni correction.

3.2.1 Questionnaire

Figure 6 showed the results of the questionnaire for Experiment 2.

Figure 6

Figure 6. Results of the questionnaire for all participants in Experiment 2.

Regarding questions Q1 to Q3 about the sense of body ownership, there were no differences between Q1 and Q3 on the question about the sense of body ownership in either condition. However, in Q2, participants were more likely to feel the sense of body ownership toward the two to eight hands in the Attention Sync condition than in the Attention Async (p = 0.048) and No Attention Async (p = 0.011) conditions. Furthermore, they also felt the sense of ownership for two to eight hands in the No Attention Sync condition than in the Attention Async (p = 0.018) and No Attention Async (p = 0.041) conditions.

Regarding questions Q4 through Q6 about the sense of agency, there was no difference between Q4 and Q5 in any of the conditions. For Q6, participants were more likely to feel the sense of agency toward the nine hands in the Attention Sync condition than in the Attention Async (p = 0.015) and No Attention Async (p = 0.022) conditions. They also felt the sense of agency for the nine hands in the No Attention Sync condition than in the Attention Async (p = 0.026) and No Attention Async (p = 0.015) conditions.

We summarized Q7 and Q8 for multiple hands perception. In Q7, no differences were found in either condition, and participants did not perceive any of the conditions as one significant move. In Q8, they recognized multiple hands more strongly in the Attention Sync condition than in the Attention Async (p = 0.0056) and No Attention Async (p = 0.0012) conditions. They also recognized multiple hands more strongly in the No Attention Sync condition than in the Attention Async (p = 0.012) and No Attention Async (p = 0.0014) conditions.

Regarding the threat stimulus questions Q9 to Q11, there were no differences between Q10 and Q11 in any of the conditions. In Q9, participants were more likely to feel threatened by one hand in the Attention Sync condition (p = 0.034) and the No Attention Sync condition (p = 0.047) than in the No Attention Async condition. On the other hand, they were not more likely to feel threatened by one hand in the Attention Sync condition than in the Attention Async condition (p = 0.23). Therefore, the single knife was a weak threat to elicit a startle response.

No difference was found in the conditions for questions Q12 Q14 regarding switching attention to multiple hands.

3.2.2 Jerk for the startle response

Figure 7 showed the results of the jerk for the startle response in Experiment 2. Participants responded more intensely to the threat stimulus in the No Attention Sync condition than in the Attention Async condition (p = 0.0014).

Figure 7

Figure 7. Results of the jerk for all participants in Experiment 2. Error bars represent 95% confidence intervals.

4 Discussion

4.1 Body perception for multiple hands

In this study, we investigated body perception in different numbers of hands in Experiment 1 and body perception of multiple bodies in the synchronous condition compared to the asynchronous condition in Experiment 2. We examined the sense of body ownership, the sense of agency, and the perception of multiple bodies using subjective evaluations.

The results of the subjective evaluation in Experiment 1 showed that the nine hands condition elicited the sense of body ownership of multiple hands more than the one hand condition in synchronous and asynchronous conditions. In addition, both the sense of body ownership of two to eight hands and the sense of ownership of nine hands were more likely to be found in the one hand condition than in the nine hands condition. On the other hand, the results of Experiment 2 indicated that the sense of body ownership was elicited for the 2 to 8 hands when compared to the synchronous and asynchronous conditions for the nine hands. While previous studies on multiple bodies had found the sense of body ownership for all bodies (Heydrich et al., 2013; Guterstam et al., 2020; Miura et al., 2022), this subjective evaluation only elicited the sense of body ownership for a subset of the increased number of bodies. This was likely related to the apparent increase in the number of bodies and the fact that the reaching task forces attention to parts of multiple bodies. Therefore, the reaching task with nine hands uniformly placed induced the sense of body ownership not for all hands but for some of the multiple hands.

Regarding the sense of agency of multiple hands, the results of the subjective evaluation in Experiment 1, the sense of agency of manipulating one hand was more robust in the one hand condition than in the nine hands condition, regardless of the synchronization condition. Conversely, the sense of agency of manipulating multiple hands was obtained for the nine hands, regardless of the synchronization condition. The results of Experiment 2 showed that the sense of agency was obtained for the nine hands, as the participants rated the sense of agency of manipulating the nine hands as more intense in the synchronous condition than in the asynchronous condition.

Regarding the recognition of multiple hands, the results of Experiment 1 indicated that the nine hands recognized more than one hand in multiple locations. In addition, the results of Experiment 2 showed that the synchronous condition of nine hands was more clearly able to recognize multiple hands than the asynchronous condition.

These results on the sense of body ownership, agency, and body perception indicated that the nine hands elicited different body perceptions than the single hand and that having multiple bodies’ ownership is possible. In particular, the sense of body ownership was perceived for 2 to 8 hands in the synchronous condition, indicating that the sense of ownership was not evoked simultaneously for all hands but rather that the sense of ownership of multiple hands was switched.

4.2 Startle response by knives to multiple hands

In this study, we investigated startle responses in multiple hands based on the jerk results and subjective ratings of Experiment 1.

The results of the startle response to the jerk in Experiment 1 showed that the nine hands synchronous condition produced more muscular movements in response to threat stimulus than the nine hands asynchronous condition. This result indicated that the synchronous condition was associated with a startle response in the dynamicity condition and that the sense of body ownership was objectively generated for the nine hands. No significant differences were reported between the synchronization condition of nine hands and one hand, indicating that the number of hands did not alter the movements in response to the threat stimulus.

The results of the startle response to subjective ratings showed that, similar to the subjective ratings of the sense of body ownership, the nine hands felt more threatened by multiple bodies than by a single hand, regardless of the synchronous or asynchronous condition. However, in contrast to the subjective ratings of the sense of body ownership, the synchronous condition for the nine hands was rated as more threatening to all nine hands than the asynchronous condition, not to two to eight hands.

There should be a relationship between the sense of body ownership and the startle response. However, the results for the sense of body ownership and subjective threat ratings did not match. These results suggested that the subjective ratings of body ownership were likely to differ between situations in which attention is focused on multiple bodies, as in the reaching task, and situations in which attention is focused on nine hands due to the threat stimulus of the nine knives. Thus, the sense of body ownership for the nine hands varied by situation, suggesting that the sense of body ownership was likely to vary by task, threat stimulus, and other factors.

The results of the objective measures and the subjective ratings above indicated that manipulating the nine hands elicited the knife’s threat stimulus and that the startle response also generated the sense of body ownership.

4.3 Importance of attention to multiple bodies

Experiment 1 examined the importance of attention in multiple bodies based on subjective ratings by comparing multiple hands to one hand. In Experiment 2, we examined the influence of attention in multiple bodies by varying the target of the threat stimulus depending on the presence or absence of attention.

We investigated the switching of the subjective sense of body ownership, attention, and sense of agency by Q12, Q13, and Q14. The results of Experiment 1 showed that the nine hands, regardless of the synchronization condition, caused more switching in the sense of body ownership, attention, and sense of agency than the single hand. There was no difference in the intensity of switching in sense of body ownership, agency, and attention between the synchronous and asynchronous conditions of the nine hands. Similarly, the results of Experiment 2 showed no difference in the intensity of switching between the sense of body ownership, sense of agency, and attention between the synchronous and asynchronous conditions for the nine hands. Based on these results of Experiments 1 and 2, we were not able to conclude about the effect of attention according to the synchronous condition in the nine hands manipulation. Therefore, we assume that the effect of attention was manifested in the nine hands manipulation itself, and that the effect of attention appeared in the handling of multiple bodies compared to the handling of a single body. Thus, the results indicated that switching bodily senses occurs subjectively independent of the synchronous condition in handling multiple bodies.

The importance of attention in multiple bodies is examined based on threat ratings. The results of the startle response to the jerk in Experiment 2 showed that the threat stimulus condition for the non-attentive hand in the synchronous condition produced more muscular movements in response to the threat stimulus than the threat stimulus condition for the attentive hand in the asynchronous condition. However, no significant differences were reported in the subjective ratings of these conditions, which is contradictory. Considering the factors that led the subjects to move their hands firmly despite the subjective absence of threat, it was possible that the subjects responded to the threat stimulus using their peripheral vision. Peripheral vision is able to detect sudden movements faster than central vision (Mckee and Nakayama, 1984; Carrasco et al., 2003). In this study, the threat stimulus condition was randomly assigned to hands that had not completed the task. Therefore, the peripheral vision perceived the threat stimulus, and this was likely a factor that elicited a more robust response to the threat.

The results of Experiment 2 subjective threat ratings showed that participants felt more threatened by one hand in each of the nine synchronous conditions than in the asynchronous condition of threat stimulus to the hand to which they were not paying attention. There was no difference in the threat stimulus condition for the hand to which attention was directed in the synchronous and asynchronous conditions, respectively. Thus, threat stimulus to the hand to which attention was not directed weakened subjective threat ratings. These results indicated that the subjective sense of body ownership was likely less intense for the hand in the inattentive condition.

Based on the results of subjective ratings of attention in the nine hands compared to the one hand and subjective threat ratings with and without attention in the nine hand manipulation, switching attention was influential in the nine hand manipulation. Furthermore, the sense of body ownership was subjectively perceived for two to eight hands in the nine hands manipulation. The results of these subjective ratings support the study (Miura et al., 2022) that reported the possibility that switching attention to sensory information between bodies is necessary to perform intentional behaviors, in that the manipulations were performed by switching attention to sensory information between bodies for multiple bodies. In addition, the effect of attention switching on the manipulation of nine hands compared to one hand and the perceived subjective sense of body ownership for two to eight hands seems to be related to a study (Jenkinson et al., 2013) that reported a more potent effect of the rubber hand illusion when attention was directed to the rubber hand, in terms of the effect of attention. Thus, the switching of attention affected the sense of body ownership for multiple bodies that increased in number.

5 Limitation

5.1 Experimental design

Our experimental design examined body perception for nine hands in a reaching task. The task required attention to be directed to the sphere, and inevitably, only several hands were in view and attended to. Therefore, it was concluded that the sense of body ownership was switched. On the other hand, the number of bodies that elicited the sense of body ownership varied depending on the situation, as in the case where the subjects felt threatened by nine hands in response to nine knives. The experimental results suggested that the number and arrangement of hands, the task, and the associated body movements change the sense of body ownership of multiple bodies. Furthermore, tactile feedback (Jahanian Najafabadi et al., 2023a; Jahanian Najafabadi et al., 2023b) and finger tracking (Adkins et al., 2021) are possible ways to enhance the sense of body ownership in the experimental design. This experiment partially examined the sense of body ownership for multiple bodies. Therefore, it is necessary to design experiments according to the questions to be investigated.

5.2 Assessment of threat stimuli

In this study, the threat stimulus for the hand not paying attention was given to a random hand that had not completed the task. Therefore, it was impossible to consider the distance from the actual hand or to control for the last hand touched by each participant. In the Rossetti et al. (2015) study, the tool-enhanced body reduced the threat response to positions away from the body. Therefore, future studies should examine the sense of body ownership by controlling which hand is used for reaching tasks and which is used for threat responses.

6 Conclusion

In this study, we examined body perception in a virtual environment toward nine hands arranged in a planar pattern visible from a first-person perspective. The body, synchronization, and threat stimulus conditions were varied, with subjective ratings by questionnaires and objective threat ratings by the jerk. The results revealed that the nine hands were subjected to body perception as multiple hands, indicating that the importance of attention and the switching of attention induced the sense of body ownership of several hands. This experiment showed that participants performed body perception for multiple virtual bodies compared to a single hand. Since multiple hands have the potential to provide various extensions, such as parallel tasks and real-world interactions, our findings are expected to contribute to the future understanding of the perception of multiple bodies.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Ethics statement

The studies involving humans were approved by Research Ethics Committee of the Faculty of Science and Technology, Keio University. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

YM: Conceptualization, Data curation, Formal Analysis, Investigation, Methodology, Resources, Software, Validation, Visualization, Writing–original draft, Writing–review and editing. RK: Conceptualization, Methodology, Writing–review and editing. MF: Conceptualization, Methodology, Writing–review and editing. TT: Conceptualization, Methodology, Writing–review and editing. MS: Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Writing–review and editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This study was supported by JST ERATO, Grant Number JPMJER1701 and JSPS KAKENHI Grant Number 23K17460.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Adkins, A., Lin, L., Normoyle, A., Canales, R., Ye, Y., and Jörg, S. (2021). Evaluating grasping visualizations and control modes in a vr game. ACM Trans. Appl. Percept. (TAP) 18, 1–14. doi:10.1145/3486582

CrossRef Full Text | Google Scholar

Argelaguet, F., Hoyet, L., Trico, M., and Lécuyer, A. (2016). “The role of interaction in virtual embodiment: effects of the virtual hand representation,” in 2016 IEEE virtual reality (VR) (IEEE), 3–10.

CrossRef Full Text | Google Scholar

Armel, K. C., and Ramachandran, V. S. (2003). Projecting sensations to external objects: evidence from skin conductance response. Proc. R. Soc. Lond. Ser. B Biol. Sci. 270, 1499–1506. doi:10.1098/rspb.2003.2364

PubMed Abstract | CrossRef Full Text | Google Scholar

Botvinick, M., and Cohen, J. (1998). Rubber hands ‘feel’touch that eyes see. Nature 391, 756. doi:10.1038/35784

PubMed Abstract | CrossRef Full Text | Google Scholar

Canales, R., and Jörg, S. (2020) Performance is not everything: audio feedback preferred over visual feedback for grasping task in virtual reality.

Google Scholar

Canales, R., Normoyle, A., Sun, Y., Ye, Y., Luca, M. D., and Jörg, S. (2019) Virtual grasping feedback and virtual hand ownership.

Google Scholar

Cardinali, L., Zanini, A., Yanofsky, R., Roy, A. C., de Vignemont, F., Culham, J. C., et al. (2021). The toolish hand illusion: embodiment of a tool based on similarity with the hand. Sci. Rep. 11, 2024. doi:10.1038/s41598-021-81706-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Carrasco, M., McElreel, B., Denisova, K., and Giordano, A. (2003). Speed of visual processing increases with eccentricity. Nat. Neurosci. 6, 699–700. doi:10.1038/nn1079

PubMed Abstract | CrossRef Full Text | Google Scholar

Ehrsson, H. H., Wiech, K., Weiskopf, N., Dolan, R. J., and Passingham, R. E. (2007). Threatening a rubber hand that you feel is yours elicits a cortical anxiety response. Proc. Natl. Acad. Sci. 104, 9828–9833. doi:10.1073/pnas.0610011104

PubMed Abstract | CrossRef Full Text | Google Scholar

Flash, T., and Hogan, N. (1985). The coordination of arm movements: an experimentally confirmed mathematical model. J. Neurosci. 5, 1688–1703. doi:10.1523/jneurosci.05-07-01688.1985

PubMed Abstract | CrossRef Full Text | Google Scholar

Fribourg, R., Argelaguet, F., Hoyet, L., and Lécuyer, A. (2018). “Studying the sense of embodiment in vr shared experiences,” in 2018 IEEE conference on virtual reality and 3D user interfaces (VR) (IEEE), 273–280.

CrossRef Full Text | Google Scholar

Gallagher, S. (2000). Philosophical conceptions of the self: implications for cognitive science. Trends cognitive Sci. 4, 14–21. doi:10.1016/s1364-6613(99)01417-5

PubMed Abstract | CrossRef Full Text | Google Scholar

González-Franco, M., Peck, T. C., Rodríguez-Fornells, A., and Slater, M. (2014). A threat to a virtual hand elicits motor cortex activation. Exp. Brain Res. 232, 875–887. doi:10.1007/s00221-013-3800-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Gonzalez-Franco, M., Perez-Marcos, D., Spanlang, B., and Slater, M. (2010). “The contribution of real-time mirror reflections of motor actions on virtual body ownership in an immersive virtual environment,” in 2010 IEEE virtual reality conference (VR) (IEEE), 111–114.

CrossRef Full Text | Google Scholar

Guterstam, A., Larsson, D. E., Szczotka, J., and Ehrsson, H. H. (2020). Duplication of the bodily self: a perceptual illusion of dual full-body ownership and dual self-location. R. Soc. open Sci. 7, 201911. doi:10.1098/rsos.201911

PubMed Abstract | CrossRef Full Text | Google Scholar

Guterstam, A., Petkova, V. I., and Ehrsson, H. H. (2011). The illusion of owning a third arm. PLOS ONE 6, e17208–e17211. doi:10.1371/journal.pone.0017208

PubMed Abstract | CrossRef Full Text | Google Scholar

Hagiwara, T., Ganesh, G., Sugimoto, M., Inami, M., and Kitazaki, M. (2020). Individuals prioritize the reach straightness and hand jerk of a shared avatar over their own. Iscience 23, 101732. doi:10.1016/j.isci.2020.101732

PubMed Abstract | CrossRef Full Text | Google Scholar

Heydrich, L., Dodds, T., Aspell, J., Herbelin, B., Buelthoff, H., Mohler, B., et al. (2013). Visual capture and the experience of having two bodies – evidence from two different virtual reality techniques. Front. Psychol. 4, 946. doi:10.3389/fpsyg.2013.00946

PubMed Abstract | CrossRef Full Text | Google Scholar

Jahanian Najafabadi, A., Küster, D., Putze, F., and Godde, B. (2023a). Emergence of sense of body ownership but not agency during virtual tool-use training is associated with an altered body schema. Exp. Brain Res. 241, 1721–1738. doi:10.1007/s00221-023-06644-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Jahanian Najafabadi, A., Küster, D., Putze, F., and Godde, B. (2023b). Tool-use training in augmented reality: plasticity of forearm body schema does not predict sense of ownership or agency in older adults. Exp. Brain Res. 241, 1739–1756. doi:10.1007/s00221-023-06645-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Jenkinson, P. M., Haggard, P., Ferreira, N. C., and Fotopoulou, A. (2013). Body ownership and attention in the mirror: insights from somatoparaphrenia and the rubber hand illusion. Neuropsychologia 51, 1453–1462. doi:10.1016/j.neuropsychologia.2013.03.029

PubMed Abstract | CrossRef Full Text | Google Scholar

Kalckert, A., and Ehrsson, H. H. (2012). Moving a rubber hand that feels like your own: a dissociation of ownership and agency. Front. Hum. Neurosci. 6, 40. doi:10.3389/fnhum.2012.00040

PubMed Abstract | CrossRef Full Text | Google Scholar

Kalckert, A., and Ehrsson, H. H. (2014). The moving rubber hand illusion revisited: comparing movements and vibrotactile stimulation to induce illusory ownership. Conscious. Cognition 26, 117–132. doi:10.1016/j.concog.2014.02.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Kilteni, K., Groten, R., and Slater, M. (2012a). The sense of embodiment in virtual reality. Presence Teleoperators Virtual Environ. 21, 373–387. doi:10.1162/pres_a_00124

CrossRef Full Text | Google Scholar

Kilteni, K., Normand, J.-M., Sanchez-Vives, M. V., and Slater, M. (2012b). Extending body space in immersive virtual reality: a very long arm illusion. PloS one 7, e40867. doi:10.1371/journal.pone.0040867

PubMed Abstract | CrossRef Full Text | Google Scholar

Lougiakis, C., Katifori, A., Roussou, M., and Ioannidis, I.-P. (2020). “Effects of virtual hand representation on interaction and embodiment in hmd-based virtual environments using controllers,” in 2020 IEEE conference on virtual reality and 3D user interfaces (VR) (IEEE), 510–518.

CrossRef Full Text | Google Scholar

Maselli, A., and Slater, M. (2013). The building blocks of the full body ownership illusion. Front. Hum. Neurosci. 7, 83. doi:10.3389/fnhum.2013.00083

PubMed Abstract | CrossRef Full Text | Google Scholar

Mckee, S. P., and Nakayama, K. (1984). The detection of motion in the peripheral visual field. Vis. Res. 24, 25–32. doi:10.1016/0042-6989(84)90140-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Miura, R., Kasahara, S., Kitazaki, M., Verhulst, A., Inami, M., and Sugimoto, M. (2022). Multisoma: motor and gaze analysis on distributed embodiment with synchronized behavior and perception. Front. Comput. Sci. 4, 788014. doi:10.3389/fcomp.2022.788014

CrossRef Full Text | Google Scholar

Peck, T. C., and Gonzalez-Franco, M. (2021). Avatar embodiment. a standardized questionnaire. Front. Virtual Real. 1, 575943. doi:10.3389/frvir.2020.575943

CrossRef Full Text | Google Scholar

Petkova, V. I., Khoshnevis, M., and Ehrsson, H. H. (2011). The perspective matters! multisensory integration in ego-centric reference frames determines full-body ownership. Front. Psychol. 2, 35. doi:10.3389/fpsyg.2011.00035

PubMed Abstract | CrossRef Full Text | Google Scholar

Rossetti, A., Romano, D., Bolognini, N., and Maravita, A. (2015). Dynamic expansion of alert responses to incoming painful stimuli following tool use. Neuropsychologia 70, 486–494. doi:10.1016/j.neuropsychologia.2015.01.019

PubMed Abstract | CrossRef Full Text | Google Scholar

Sanchez-Vives, M. V., Spanlang, B., Frisoli, A., Bergamasco, M., and Slater, M. (2010). Virtual hand illusion induced by visuomotor correlations. PloS one 5, e10381. doi:10.1371/journal.pone.0010381

PubMed Abstract | CrossRef Full Text | Google Scholar

Schjerlund, J., Hornbæk, K., and Bergström, J. (2021). “Ninja hands: using many hands to improve target selection in vr,” in Proceedings of the 2021 CHI conference on human factors in computing systems, 1–14.

CrossRef Full Text | Google Scholar

Schone, H. R., Udeozor, M., Moninghoff, M., Rispoli, B., Vandersea, J., Lock, B., et al. (2024). Biomimetic versus arbitrary motor control strategies for bionic hand skill learning. Nat. Hum. Behav., 1–16. doi:10.1038/s41562-023-01811-6

CrossRef Full Text | Google Scholar

Slater, M., Pérez Marcos, D., Ehrsson, H., and Sanchez-Vives, M. V. (2008). Towards a digital body: the virtual arm illusion. Front. Hum. Neurosci. 2 (6), 6. doi:10.3389/neuro.09.006.2008

PubMed Abstract | CrossRef Full Text | Google Scholar

Slater, M., Spanlang, B., Sanchez-Vives, M. V., and Blanke, O. (2010). First person experience of body transfer in virtual reality. PloS one 5, e10564. doi:10.1371/journal.pone.0010564

PubMed Abstract | CrossRef Full Text | Google Scholar

Tsuji, T., Yamakawa, H., Yamashita, A., Takakusaki, K., Maeda, T., Kato, M., et al. (2013). “Analysis of electromyography and skin conductance response during rubber hand illusion,” in 2013 IEEE workshop on advanced robotics and its social impacts (IEEE), 88–93.

CrossRef Full Text | Google Scholar

Venkatakrishnan, R., Venkatakrishnan, R., Raveendranath, B., Pagano, C. C., Robb, A. C., Lin, W.-C., et al. (2023). How virtual hand representations affect the perceptions of dynamic affordances in virtual reality. IEEE Trans. Vis. Comput. Graph. 29, 2258–2268. doi:10.1109/tvcg.2023.3247041

CrossRef Full Text | Google Scholar

Woods, D. L., Wyma, J. M., Yund, E. W., Herron, T. J., and Reed, B. (2015). Factors influencing the latency of simple reaction time. Front. Hum. Neurosci. 9, 131. doi:10.3389/fnhum.2015.00131

PubMed Abstract | CrossRef Full Text | Google Scholar

Yuan, Y., and Steed, A. (2010). “Is the rubber hand illusion induced by immersive virtual reality?,” in 2010 IEEE virtual reality conference (VR) (IEEE), 95–102.

CrossRef Full Text | Google Scholar

Zhang, J., Ma, K., and Hommel, B. (2015). The virtual hand illusion is moderated by context-induced spatial reference frames. Front. Psychol. 6, 1659. doi:10.3389/fpsyg.2015.01659

PubMed Abstract | CrossRef Full Text | Google Scholar