Weight perception analysis using pseudo-haptic feedback based on physical work evaluation

Introduction: Since kinesthetic cues are not present in virtual environments, users have difficulty feeling the heaviness of virtual objects. To address this issue, pseudo-haptic approaches have been proposed to illusorily induce the weight of virtual objects through the user’s visual sensory system.

Methods: In this paper, we used two methods to induce the impression of virtual objects’ heaviness. One relies on the direct modification of the control-display (C/D) ratio when lifting objects, and the other depends on controlling this ratio based on a velocity restriction. We innovatively measured each approach’s efficiency by analyzing physical work as an objective metric. In addition, we used Borg CR10 to measure users’ hand fatigue during the experimental phases.

Results: Our findings are discussed in terms of individual lifting behavior in different pseudo-haptic methods. Furthermore, different virtual weight-lifting behaviors were compared to the same real-world weight-lifting behaviors.

Discussion: According to our results, the direct control of the C/D ratio method provides virtual reality users with a more accurate weight perception than the velocity restriction one. Furthermore, with this first method, users’ lifting behavior was closer to the behavior when lifting real objects.

1 Introduction

One of the significant challenges and research topics in virtual reality (VR) is to enhance the user experience so that users can feel as if they were in a real environment. While many studies have been conducted on this topic (Nguyen and Bednarz, 2020), open problems still remain regarding creating a rich and believable virtual environment (VE). One crucial modality is haptics, which is typically delivered to VR users through haptic feedback (Burdea, 1996). It allows users to select and manipulate objects in a more realistic manner (Ramsamy et al., 2006). Consequently, it gives users a greater sense of presence (Gibbs et al., 2022).

By grasping and manipulating an object, humans obtain certain information about it, such as its size, shape, and texture. This information is mainly provided by the vision and haptic senses (Ernst and Banks, 2002). Since visual perception occurs exclusively through the retina, visual information is unimodal. Alternatively, haptic perception involves several sources of information, including force feedback, proprioceptive feedback, and tactile feedback (Aman et al., 2010). A single, unified haptic perception is thus obtained by combining such information (Sciutti et al., 2010) (Burdea, 1996) (Burdea, 1999). Humans can recognize weight, inertia, and object hardness based on force feedback, while proprioceptive feedback provides information about body position, and tactile feedback is used to distinguish object surface (Burdea, 1999) (De Tinguy et al., 2018).

Regarding users’ interactions in VEs, visual feedback is usually fully supplied through the devices used (VR head-mounted display (HMD), mixed reality glasses, etc.). However, regarding other senses, the virtual environment does not provide this feedback, or elements from the real world can interfere (e.g., ambient noise vs. displayed sound in the HMD). Therefore, a conflict happens between visual information and other sources of sensory information, especially haptic sensors. In addition, the mismatch between the real and virtual environment leads to difficulty in enabling true haptic feedback (Rietzler et al., 2018), particularly concerning the kinesthetic feedback, which relies on the physical direction of the force and therefore is difficult to compensate for (Rietzler et al., 2018). However, about other aspects of the haptic sense, some methods, such as vibration stimuli techniques, have been introduced in the literature and they succeeded in providing tactile feedback sensation. Regarding force feedback, when users perceive an object through their visual sensory system and want to interact with it and move it, they are usually unable to truly grasp or lift it since the object is not real. Thus, they do not correctly perceive its weight. Because of this, in the real world, users cannot perceive their own relative tiredness of lifting such objects in their muscles.

However, perceiving the heaviness of objects cannot be limited only to our haptic sensory system; it also relies on our visual sensory system (Runeson and Frykholm, 1981). Therefore, in VEs, approaches such as the “pseudo-haptic feedback” technique have been developed to benefit from the visual sensory system when evaluating virtual objects heaviness. Such approaches propose to induce heaviness by hacking human perception through visual stimuli, playing on the borders of human perception and sensory illusions. With this illusory method, users can enjoy a haptic experience induced only by visual stimuli only (Lécuyer, 2009). Previous studies have shown that a VR user’s experience is improved by using this technique, making it feel closer to a real-world interaction (Yu and Bowman, 2020) (Ujitoko and Ban, 2021).

Even though this method is one of the most effective options for replacing costly and expensive haptic devices, it still requires significant improvement. One of the main challenges of this method is measuring the sense of presence, particularly when subjects are dealing with heavy objects. One particular issue is the need to evaluate and measure the sense of presence when using it, especially when subjects deal with heavy objects. Many studies (Rietzler et al., 2018), (Maehigashi et al., 2021), (Lécuyer et al., 2004), (Zenner and Krüger, 2017) focused on virtually induced weights of objects to VR participants focused on the use of subjective measurements to evaluate their pseudo-haptic methods. One issue regarding such measures is that they depend on the subject’s point of view and rely only on intuition. However, few other studiesfocused on objective measurements: in particular, Samad et al. (2019) used the concept of work to evaluate their model, in the context of lifting light small cubes (185 g). One limitation of this work is that in real environments, humans also manipulate heavier objects, about at least 1 kg and up to 5 kg, in everyday life.

We propose and evaluate different pseudo-haptic feedback techniques, used in the context of a lifting task in VR. We build our techniques relying on anisomorphic mapping, i.e., based on a difference between the motions performed by users in the real environment and those they observe in the VE. With these techniques, we allow users to lift “heavy” virtual objects. We propose two different pseudo-haptic feedback models. One is based on applying a determined control-distance (C/D) ratio to the object’s displacement, and the other modifies the motion by limiting the maximum displacement speed of the object, both models being linked to the same (C/D) ratio concept. Finally, we simulate different virtual weights with both techniques in our experiments. Furthermore, we consider a control condition (lifting real objects) in which participants raise real objects with actual weights. Finally, we also provide the participants with a lifting technique without haptic or pseudo-haptic feedback as a control condition (isomorphic movements).

We propose a mechanism that can diminish visuo-kinesthetic conflicts in VR during lifting tasks and convey the concept of weight to VR users. Additionally, we offer to evaluate such an approach by measuring both the subjective effects of this mechanism (measuring fatigue) and the objective effects (physical activity, in terms of work) of VR users. As a result, this study makes the following contributions:

• Designing different manipulation conditions in VR, based on the physical work of expected/targeted objects’ weight, to be induced in VR on virtual objects.

• Studying the effect of pseudo-haptic feedback on VR users’ lifting behaviors and comparing them to lifting real objects.

• Extending previous work results from lighter objects (less than 0.5 kg) to heavier objects (1, 2, and 5 kg).

• Formalizing the notion of physical work to study different individual behaviors in virtual and real environments by using it as an objective measure of the sense of presence.

• Evaluating relationships regarding the sense of presence between objective (physical work) and subjective measurements (sensation of fatigue).

2 Related work

2.1 Manipulation interaction techniques

Bowman and Hodges (1999) classified interactions into three categories: navigation, selection, and manipulation. A taxonomy was developed for each type of VE interaction. Regarding on Bowman’s taxonomy about manipulation interaction techniques, the techniques are differentiated based on several criteria, such as the way to attach an object to the user or the way to move the object (translation and rotation). In a similar manner, Poupyrev et al. (1998) (Poupyrev and Ichikawa, 1999) evaluated manipulation techniques based on a variety of criteria: exocentricity (users act as if they are outside the environment) or egocentricity (users act as if they are inside the environment). Bowman et al. and Poupyrev et al. determined that the selection and manipulation of interaction techniques could be built in a similar manner and shared many criteria. The selection techniques studied by Argelaguet and Andujar (2013) are very relevant to understanding manipulation interaction techniques, such as selection tools (e.g., hand, ray, cone) and the C/D ratio. Generally, the C/D is defined as the ratio between the input devices’ translational motion and the selection tools’ translational motions. The selection technique is called isomorphic when the C/D ratio is equal to one; otherwise, it is anisomorphic - either scaled up (<1) or down (>1).

These different taxonomies for selection and manipulation techniques (Bowman and Hodges, 1999) (Bowman et al., 2001) (Poupyrev et al., 1998) (Poupyrev and Ichikawa, 1999) (Argelaguet and Andujar, 2013) suggest that two main criteria should be considered when designing an interaction technique. First, the manipulation support (3D hand, raycast, etc.), and second, the nature of the mapping between the real and virtual movements. It has been demonstrated that C/D ratios different than one can be implemented for various selection and manipulation techniques. These techniques could be either using virtual hands, as with the Go-Go technique (Poupyrev et al., 1996), or using raycast, as with the PRISM technique (Frees et al., 2007) and the virtual pads technique (Andujar and Argelaguet, 2007). These studies utilized different C/D ratios in order to optimize interaction (for instance in downscaling the C/D ratio to provide more precision). However, few studies have attempted to provide pseudo-haptic feedback to users using anisomorphic manipulation techniques.

2.2 Haptic and pseudo-haptic feedback

Simulating a virtual objects’ weight is challenging, as no kinesthetic cues are present in such environments. Researchers have traditionally concentrated on grounded haptic devices such as the Phantom device to overcome this problem (Pacchierotti et al., 2017), (Burdea, 1999), (Massie and Salisbury, 1994). In spite of the advantages of these devices, such as their dynamic range and degrees of freedom (Nisar et al., 2018), their main disadvantages are their complexity, limited workspace, and cost (Pacchierotti et al., 2017) (Samad et al., 2019). Regarding movable haptic devices, numerous portable and wearable haptic devices provide significant range of motion; non-etheless, these mostly concentrate on tactile feedback. Such devices have mainly been designed to deliver tactile feedback (Nisar et al., 2018). Non-etheless, it is worth mentioning that Choi et al. (2017) developed a haptic glove called Grabity, which provides touch, texture, and weight sensations (but reduces the user’s natural range of motion).

Following another approach, VR researchers have developed a pseudo-haptic technique to evoke haptic perception through visual cues (Ujitoko and Ban, 2021). As a result, they can provide haptic perception without using expensive or restrictive haptic devices. It is also consistent with findings by Ernst and Banks (2002) who found that visual-haptic perception dominates in judging objects’ shape, size, and position. For example, in the literature, Rock and Victor (Rock and Victor, 1964) asked participants to wear distorted glasses and grasp a square, while they perceived rectangles through the lenses, and their results showed the importance of vision through the users’ difficulties to grasp. Due to this, the object’s shape is mostly perceived by vision, known as “visual capture” (Ernst and Banks, 2002). These statements and results have prompted numerous VR research studies to employ the pseudo-haptic technique, namely to simulate different haptic sensations of virtual objects, including friction, stiffness, and texture (De Tinguy et al., 2018) (Lécuyer, 2009), or to simulate the heaviness of objects (Jauregui et al., 2014), (Palmerius et al., 2014), (Yu and Bowman, 2020), (Samad et al., 2019), (Lee et al., 2019).

Pseudo-haptic feedback can be provided by manipulating the C/D ratio (Poupyrev et al., 1996) (Argelaguet and Andujar, 2013). To induce friction, Lécuyer et al. (2000) designed a coupling between slowing down the velocity of the object’s movement and incrementing the reaction force coming from the device, creating some illusory force feedback. Other studies (Dominjon et al., 2005), (Nakakoji et al., 2011), (Nakakoji et al., 2010) designed experiments to induce the weight of the object using the C/D ratio approach. While these experiments were conducted in a simple 2D environment, the comparison with VE interactions with virtual objects is hard to perform. Non-etheless, these studies were able to demonstrate that the C/D ratio method is effective.

Recent studies on weight perception in VR environments using pseudo-haptic feedback have demonstrated that the pseudo-haptic method had an impact on increasing the sense of presence (Samad et al., 2019), (Rietzler et al., 2018). A close connection between this illusionary approach and a sense of presence resulted in a narrow line between increasing or eliminating the feeling of presence during the VR experience. Rosa et al. (2015) designed experiments to produce illusory weight and temperature by influencing vibrational perception with visual signals like size and speed. In light of their findings, Based on their findings, it can be stated that if the visual stimulus shows the gaining of the weight of the virtual object, and at the same time, the tactile feedback has low intensity, the illusory weight is not only intangible but also destructive to the sense of presence. It should be noted that they did not quantify the relationship between visual and vibrotactile stimuli and perception of weight in their experiment.

To conclude, according to the literature, it would be helpful to design and evaluate an isomorphic manipulation technique to improve the user experience in VR without compromising the sense of presence. In addition, in our study, we propose to link anisomorphic interaction techniques to the importance of physics when lifting objects. Indeed, variations in the C/D ratio applied to lifted objects can be linked to variations in the lifted weight.

3 Materials and methods

3.1 Overview

First, we propose an approach based on physics to compute pseudo-haptic feedback in a virtual reality environment while performing a lifting task. Second, we evaluate the responses of VR users to this feedback through an experiment. Finally, we use both objective and subjective measures to determine the effect of this feedback on the effort performed by the users and their perception of this effort.

Relying on our physical model, we developed and implemented two anisomorphic pseudo-haptic manipulation techniques. We also implemented an isomorphic manipulation technique without any feedback regarding the lifting task and object weights. Lastly, we also asked our VR users to lift real objects (with their real expected weight) within the virtual environment, thus providing real haptic feedback, as a baseline in terms of effort. Accordingly, we developed a repeated-measure experiment in which participants were required to vertically move a water carrier–that we called it in the experiment water bottle-with different masses under the four conditions described above.

3.2 Our methods for pseudo-haptic feedback: From a physics model to anisomorphic manipulations

We propose an approach based on physics to produce pseudo-haptic feedback, in which the sole opposition force to a vertically lifted object is its weight, when friction is ignored. Thus, the user must produce at least a force equivalent to the weight of the lifted object, but in the opposite direction. Traditional VR setups, however, do not provide real props that users can use to interact with virtual objects. As a result, users only need to compensate for the weight of a VR controller during such a lifting operation, in contrast to the actual weight of the virtual object being viewed. Therefore, the relationship between the force exerted by the user to lift the VR controller and the force expected based on the weight of the virtual object viewed can be expressed as a ratio function called k(m):

$k\left(m\right)=\frac{\overrightarrow{\left|\left|{\mathrm{W}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{r}}\right|\right|}}{\overrightarrow{\left|\left|{\mathrm{W}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}_{\mathrm{o}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}}\right|\right|}}=\frac{{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{s}}_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{r}}}{\mathrm{m}}(1)$

Where m is the mass of the object. Then, we propose to use this ratio function to provide pseudo-haptic feedback; it is converted into either a ratio of work (Eq. 2) or a ratio of power (Eq. 3) as follows:

$W=\overrightarrow{\left|\left|\mathrm{F}\right|\right|}\ast d\ast \cos \left(\theta \right)=\overrightarrow{\left|\left|\mathrm{F}\right|\right|}\ast d,when\theta =0(2)$

$P_{instantaneous}=\overrightarrow{\left|\left|\mathrm{F}\right|\right|}\ast v\ast \cos \left(\theta \right)=\overrightarrow{\left|\left|\mathrm{F}\right|\right|}\ast v,when\theta =0(3)$

In this equation, d is the displacement distance, v is the motion velocity, and F is the force applied. In addition, we propose to use (Eq. 2) to provide pseudo-haptic feedback in a distance-based approach and (Eq. 3) in a velocity-based approach. In our case, $\theta =0$ as the movement is on the vertical axis.

3.2.1 Distance-based approach: Our “direct-weight” technique

From (Eq. 1), (Eq. 2) transforms to the following:

$\frac{Wcontroller}{WObject}=\frac{\overrightarrow{\left|\left|{\mathrm{W}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{r}}\right|\right|}\ast d_{controller}}{\overrightarrow{\left|\left|{\mathrm{W}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}_{\mathrm{o}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}}\right|\right|}\ast d_{object}}=k\left(m\right)\ast \frac{d_{controller}}{d_{object}}$

Furthermore, the condition of feedback that fully compensates for the difference in mass between the object and the controller implies a ratio of work equal to 1:

$\frac{Wcontroller}{WObject}=1\iff k\left(m\right)\ast \frac{d_{controller}}{d_{object}}=1$

$\iff d_{object}=k\left(m\right)\ast d_{controller}\iff d_{object}=\frac{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{r}}{\mathrm{m}}\ast d_{controller}$

To allow ranges of masses for our virtual objects that can be easily more than ten times the mass of the VR controller, a constant c can be added in the previous equation as follows:

$d_{object}=\left(\frac{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{r}}{\mathrm{m}}+c\right)\ast d_{controller}(4)$

This constant also limits the ratio between the two distances since the mass ratio tends to zero as m increases. Through such a limit, pseudo-haptic feedback is prevented from being applied when it would introduce such an excessive difference between real and viewed displacements. This limitation would cause the VR user to lose the feeling of presence. Additionally, it determines the minimum mass for obtaining some pseudo-haptic feedback, that is, the mass for which the ratio applied is 1, as shown below:

$\frac{d_{object}}{d_{controller}}=1\iff \frac{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{r}}{{\mathrm{m}}_{\min }}+c=1\iff {\mathrm{m}}_{\min }=\frac{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{r}}{1-c}(5)$

Lastly, we used the relationship defined in (Eq. 4) between distances to generate pseudo-haptic feedback in VR based on the masses of the virtual object and the controller. In VR, the object is scaled down compared to the distance traveled by the controller when using a C/D ratio greater than 1. In this paper, this pseudo-haptic technique is referred to as the direct-weight method (direct modification of C/D ratio, related to weight ratio), which is based on distances.

3.2.2 Velocity-based approach: Our “speed-control” technique

From (Eq. 1), (Eq. 3) transforms to the following:

$\frac{{\mathrm{P}}_{{\mathrm{i}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{e}\mathrm{o}\mathrm{u}\mathrm{s}}_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{r}}}}{{\mathrm{P}}_{{\mathrm{i}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{e}\mathrm{o}\mathrm{u}\mathrm{s}}_{\mathrm{o}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}}}}=\frac{\overrightarrow{\left|\left|\mathrm{W}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{r}\right|\right|}\ast v_{controller}}{\overrightarrow{\left|\left|\mathrm{W}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}\mathrm{O}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\right|\right|}\ast v_{object}}=k\left(m\right)\ast \frac{v_{controller}}{v_{object}}$

Further, conditions of feedback that fully compensates for the difference of masses between the object and controller mean a ratio of power equal to 1:

$\frac{{\mathrm{P}}_{{\mathrm{i}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{e}\mathrm{o}\mathrm{u}\mathrm{s}}_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{r}}}}{{\mathrm{P}}_{{\mathrm{i}\mathrm{n}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{e}\mathrm{o}\mathrm{u}\mathrm{s}}_{\mathrm{o}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}}}}=1\iff k\left(m\right)\ast \frac{v_{controller}}{v_{object}}=1$

$v_{object}=k\left(m\right)\ast v_{controller}{\iff v}_{object}=\frac{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{r}}{\mathrm{m}}\ast v_{controller}$

For the same reason as in Section 3.2.1, a constant c is used here to allow a greater range of mass values for the mass m:

$v_{object}=\left(\frac{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{r}}{\mathrm{m}}+c\right)\ast v_{controller}(6)$

As a result, the following algorithm is applied for each determined mass m:

$if{\mathrm{V}}_{controller}<{\mathrm{V}}_{{\max }_{\mathrm{o}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}}},then{v}_{object}=v_{controller}(\mathrm{7A})$

$else{v}_{object}={\mathrm{V}}_{{\max }_{\mathrm{o}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}}}/v_{controller}(\mathrm{7B})$

Finally, we used the relation defined in (Eq. 6), (Eq. 7A), and (Eq. 7B) between velocities to describe some pseudo-haptic feedback in VR depending on the mass of the virtual object and the controller; it consists of applying a C/D ratio higher than 1, defined by the opposite of $\left(\frac{\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{l}\mathrm{l}\mathrm{e}\mathrm{r}}{\mathrm{m}}+c\right)$ as in Section 3.2.1, but this time only when the velocity of the real motion is above a determined maximum speed. In this paper, we called this pseudo-haptic technique, based on velocity restriction, the speed-control technique (modification of C/D ratio related to weights ratio, limiting to a maximum speed related to object mass).

3.3 Stimuli creation

3.3.1 Virtual objects, real objects, and VR controller

In our experiment, we asked users to lift virtual water bottles under three conditions and real water bottles in one control condition (a real-world condition with the real weight, but still in VR, to prevent external differences between experimental conditions). This choice of water bottle was made because of its practicality to have multiple and identical real objects in terms of shape, but with possible different masses. In addition, it was easy to track such an object, with a HTC Vive tracker fixed on it. Moreover, water bottle grips can be easily grabbed by users, even when immersed in VR. Figures 1A, B show a real and virtual bottle respectively, with their tracker attached. Regarding the VR controller, an HTC Vive controller with a mass of 308 g was used, as shown in Figure 1C.

FIGURE 1

FIGURE 1. Real bottle (A) and virtual bottle (B) representations in our experiment, and (C) their tracker, the HTC Vive VR controller. Mass of 308 g, lifted by users during their task in VR.

3.3.2 Incongruous lifting conditions: Techniques with pseudo-haptic feedback

Our two techniques with pseudo-haptic feedback (direct-weight and speed-control), both anisomorphic manipulations, could also be called incongruous conditions. Indeed, they were both designed to induce a conflict between the visual feedback of the motion, represented in VR by the bottle displacement, and the actual arm motion, shown in VR by the controller displacement. Figure 2 shows such a difference regarding the positions between the two elements during the lifting task. In these conditions, VR users could observe the bottle moving precisely in the same direction and orientation that they moved their hand, but with a difference in speed or distance concerning their natural velocity/position.

FIGURE 2

FIGURE 2. Incongruous conditions: visual discrepancy caused by differences in position/velocity between the controller and the object, highlighted in yellow here.

3.3.2.1 C/D ratio, visual discrepancy, and objects masses

First, as explained by (Eq. 5) and as used in our two techniques through (Eq. 4) and (Eq. 6), a constant c had to be defined concerning the chosen experimental conditions and not with the physics model itself. In our experiment, we set c to a value of 0.5 for the following reasons and implications:

1) The constant of 0.5 makes the functions d(m) and v(m) in (Eq. 4) and (Eq. 6) tend towards 0.5. This avoids distortions in the presence sensation caused by a visual discrepancy between real and virtual movements.

2) Regarding objects’ masses, a value of 0.5 gives a minimum mass (m_min) of 0.616 g according to (Eq. 5) for a 0.308 g VR controller. Thus, it would mean that pseudo-haptic feedback would start for masses above 0.616 g, with ratios decreasing then from one to 0.5. This would allow for interesting intermediate points at 1 and 2 kg and a 5 kg point that would be already close to the 0.5 limits. As a result, the masses used in our experiment were fixed to 0.616, 1, 2, and 5 kg. In addition, these values would allow a significant evaluation of our pseudo-haptic techniques with masses that are already well beyond the 0.308 gr of the VR controller and close to the masses of many everyday objects.

3.3.2.2 Direct-weight condition

Regarding this first incongruous condition, apart from the constant c and considered objects’ masses, no additional parameter was required to be set for our experiment.

3.3.2.3 Speed-control condition

For this second condition, in addition to the constant c and masses of the objects, the ${\mathrm{V}}_{{\max }_{\mathrm{o}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}}}$ parameter had to be set for our experiment according to (Eq. 7A and Eq. 7B), determined for a lifting task in VR without any extra mass. If this value could be measured empirically with some users in a pre-experiment, it could also be estimated theoretically, based on Fitts’ Law studies and notably K. T. Hagadorn’s (Hagadorn, 2004). The originality of these values explains the Fitts’ law applicability for human movement in three dimensions, in the manipulation technique of moving objects, instead of being related to 2D Fitts’ law with selection or pointing tasks (Gillan et al., 1990) (MacKenzie, 1993). Additionally, this study also found that different Fitts’ laws could exist depending on the mass of the manipulated object. Therefore, we used the coefficients given in this study for objects of less than 450 g, i.e., a = 0.2138 and b = 0.473, in their formula, as follows:

$\mathrm{M}\mathrm{T}=\mathrm{a}+\mathrm{Log}2\left(2\ast \mathrm{d}/\mathrm{w}\right)\ast \mathrm{b}(8)$

where MT is the task completion time, d the manipulation distance, and w the size of the target to reach.

The distance in our experiment was 1 m, and the target size was 0.4 m, yielding an MT of 1.32 s, which implies an average speed of 0.75 m/s. From this average speed, we then needed to compute the maximum speed during a vertical lifting task. For that, we captured a lifting task motion and computed the instantaneous speed, as shown in Figure 3. Due to strength and other individual variability, the maximum speed value cannot be directly considered as our experiment’s value. However, the shape of the velocity profile for this type of task can be maintained and analyzed (the same across multiple users). Such motion can be decomposed in terms of velocity phases into acceleration and deceleration, both representing half of the movement. From that, the maximum speed can be computed from the average, by representing this velocity profile with two affine functions, f1 (ascending) and f2 (decreasing), as follows:

${\mathrm{V}}_{average}=\frac{1}{2}\ast {\int }_0^{0.66}f1\left(t\right)+\frac{1}{2}\ast {\int }_{0.66}^{1.33}f2\left(t\right)$

$\iff {\mathrm{V}}_{average}=\frac{1}{2}\ast \left(\frac{{0+v}_{\max }}{2}\right)+\frac{1}{2}\ast \left(\frac{v_{\max }+0}{2}\right)\iff {\mathrm{V}}_{average}=\frac{1}{2}\ast v_{\max }\iff {\mathrm{V}}_{\mathit{max}}=2\ast v_{\mathrm{a}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{g}\mathrm{e}}(9)$

Thus, for our experiment, we found and set the maximum speed to 1.5 m/s (0.75 × 2) for masses below or equal to 0.616 g, and for higher masses, we computed it from (Eq. 6), based on this value.

FIGURE 3

FIGURE 3. Hand motion instantaneous speed across time for a lifting task (1 m distance, no extra mass).

3.3.3 Congruous conditions: Absence of pseudo-haptic feedback–no weight at all or real weight

Besides the two conditions described in the previous section, our experiment also included the following two congruent conditions without visual discrepanc. When visual feedback aligned with the actual hand motion, the condition named “isomorphic condition”. This condition refers to “traditional” virtual manipulation technique with no haptic feedback and thus the same visual input and natural motions for any masses lifted and the other condition is “haptic condition,” which is refer to the “traditional” real manipulation technique, with full haptic feedback by lifting a real object with its actual expected mass.

3.4 Apparatus

This study required users to stand throughout the whole experiment when lifting the objects from bottom to top. However, they were permitted to rest at any time, especially between blocks of four conditions. Physically, they were placed in a room facing a 46 cm high box at a distance, allowing them to comfortably grasp and lift the objects placed on the box without bending. The virtual environment displayed within the VR headset shared the same characteristics (room size, user’s position, orientation, and relative distance to the objects to lift)—see Figure 4A.

FIGURE 4

FIGURE 4. (A) Virtual environment displayed in the VR HMD. (B) User’s view before starting to lift the bottle, grabbing the bottle (visual feedback in blue). (C) View of the end of the task (bottle in green).

A HTC Vive Pro VR headset was used for our experiment, equipped with two cameras, hand controllers to manipulate virtual objects, and hand trackers to record arm movements. The HTC vive pro has a1440 × 1,600 pixel resolution with a 110° field of view. This HMD featured, an electronic gyroscope, and an eye comfort setting system (IPD).

The hand controller provides an indication of the position of the subject’s dominant hand (left or right) in the real environment, allowing the rendering engine to generate a visual representation of the hand in VR in all conditions that involved virtual objects to lift. Hand trackers for lifting real objects, attached to the wrist of the subjects, provide locational information to the rendering engine. This information is used in order to generate a model of the subject’s hand and the visual feedback about the position of the user’s dominant hand in haptic condition. Additionally, one tracker was placed on the real object to track its displacement and to display it in the virtual environment accordingly, as shown in Figures 4B, C.

3.5 Participants

Twenty right-handed users (6 women and 14 men, ranging from 18 to 44 years with a mean age of 26.24 ± 7.98 SD) participated in our experiment. All were healthy and had no neurological, muscular, or cognitive disorders, with normal or corrected-to-normal vision. Users’ heights ranged from 156 to 185 cm, with a mean height of 173.74 ± 6.57 cm. Users from different backgrounds, either from inside or outside the university, agreed to participate voluntarily without compensation.

3.6 Experimental procedure

We conducted two phases of our experiment: the training and the main phases. In both phases, user task was to lift an object upward with a single joint arm movement, with natural self-selected speed (with rotation around the shoulder and maintaining the arm entirely extended). This object was visible in the environment as a water bottle and had to be placed at a defined height, represented by a green window (see Figure 4).

The training phase was meant to introduce users to our unusual manipulation techniques, compared to real-world lifting, notably for the incongruous conditions that cause conflicts between the actual hand movement and the given visual feedback. It would then help users to avoid “failing” to accomplish their lifting motion in terms of performance and “naturalness”. In this phase, eight lifting movements are performed before each manipulation interaction condition, in order to teach users how to perform upward lifting in our VR setup.

The main phase was divided into four blocks of lifting trials, one for each manipulation condition. With all our techniques, grabbing was always done through direct contact with objects, using a virtual hand metaphor. Each block included 20 trials, each consisting of five repetitions of lifting objects of four different masses (0.616, 1, 2, and 5 kg); see Section 3.3.2 for more details. In all trials, users performed vertical arm motions almost exclusively at a distance of 1 m, starting from a similar point (46 cm from the ground). The mass of the objects was never disclosed to the users. Furthermore, the virtual objects displayed all had the same design, without any variation in size or color. The users were asked to perform natural and self-selected-speed movements under all conditions, with congruent conditions more favourable because of the absence of visual discrepancy, and incongruous conditions more challenging due to pseudo-haptic feedback.

We used a repeated-measure design to increase the number of measures and control differences between users, as they are usually not equal in terms of strength A Latin-square order was used between participants, with the order of blocks within each block, i.e., between masses of objects, randomized for each participant. Thus, we used 20 of the 24 possible orders across our four manipulation conditions.

3.7 Data gathering and measures

3.7.1 Kinematic features and physical work

In order to collect kinematic data on hand position and velocity, we tracked the VR hand controller in non-haptic conditions and the VR hand tracker in haptic conditions. Next, we applied a low-pass filter (Butterworth) with a cut-off frequency of 6 Hz to the velocity data.

From this data, we extracted the following parameters: 1) movement duration (MD): the time between lifting onset and termination; and 2) displacement (disp): the whole vertical displacement of the user’s real hand when lifting the bottle. In addition, we computed the physical work done by the users to perform the lifting task using the following formula:

$W=m_{controller}Gh+m_{controller}\ast {\int }_s^e\alpha dx(10)$

This formula comes from the general principle of physics about forces, as expressed by the following:

${\sum }_{1=1}^{n}F=m_{controller}\ast a(11)$

Where $m$ is the controller’s mass, and $a$ is the acceleration caused by users’ movement. Then, still in physics, work can be defined by the following equation at each instant:

$dw=F.dx(12)$

where $dw$ is the element of work at each time, $F$ is the force obtained from (Eq. 11), and $dx$ is the displacement. In this study, two main forces were applied to the object, as shown in Figure 5. The first one is the users’ hand force (F) applied to lift the object upward, and the second is gravity (P).

FIGURE 5

FIGURE 5. Forces applied to an object during the lifting task.

F is the force applied by the participants’ hand on the object. Therefore, according to the physic base rules, we obtain

$\overrightarrow{\mathrm{F}}+\overrightarrow{\mathrm{P}}=\mathrm{m}\overrightarrow{a}\iff \overrightarrow{\mathrm{F}}-\overrightarrow{\mathrm{P}}=\mathrm{m}\overrightarrow{a}\mathrm{S}\mathrm{i}\mathrm{n}\mathrm{c}\mathrm{e}FandParenotinthesamedirection(13)$

Then, using the formula $\mathrm{P}=mg$, we can obtain the following equation along the vertical axis:

$F=mg+ma(14)$

Using (Eq. 12), we can express the work of the force applied by the hand of the users between the starting point, called (s) here, and the end point, called (e) here, as follows:

$W={\int }_S^{e}dw={\int }_s^{e}F.dx\iff W={\int }_s^{e}mg+ma\iff W={\int }_s^{e}mgdx+{\int }_s^{e}madx(15)$

Finally, from (Eq. 15), (Eq. 10) is obtained and used to compute the user’s hand work when lifting.

3.7.2 Perceived fatigue

We used the Borg CR10 (Borg, 1990) questionnaire to evaluate the effect of pseudo-haptic feedback. A modified version of it was used to compare how users perceived tiredness after different object lifting conditions. This provides a self-report measure of the perceived effort, relying on a 10-point self-report inventory with defined levels. Users were asked to give each manipulation condition a score according to its difficulty, allowing for the measure of their hand fatigue.

3.8 Hypotheses

As part of this study, we examined how pseudo-haptic feedback can enhance perception of an object’s weight in VR. We are also interested in understanding how we can create artificial tiredness in users’ arms in a manner similar to the tiredness produced by lifting real objects. Additionally, we sought to determine which pseudo-haptic techniques would enhance users’ perception of weight in VR in comparison with real lifting behaviors. A further objective was to understand the inter-subject variability caused by different simulations of weight perception models based on individual lifting behaviors. Therefore, the following hypotheses were investigated in this experiment:

• How does modifying the C/D ratio affect the physical work obtained from distance control and speed control? In order to replicate the haptic experience (real bottle lifting) in VR, we sought to determine which pseudo-haptic models could accomplish this.

• What are the effects of different proposed models on the fatigue level experienced by VR users?

• How do individuals perceive different weight lifting conditions in terms of perception (non-haptic) and execution (haptic)?

4 Results

4.1 Statistical analysis

Statistical analyses were performed with SPSS software (version 26, IBM SPSS) on all variables. A two-way analysis of variance (ANOVA) was performed on different variables (α = 0.05). The independent variables were “condition” (4 levels; haptic, isomorphic, speed-control, direct-weight) and “weight” (4 levels; 0.616, 1, 2, 5 in kg). In addition, for each condition, a one-way ANOVA test was separately applied to see the effect of real and virtual weight (independent variable) on physical work (dependent variable). To control false discovery rate, we used the Benjamini–Hochberg (B-H) model at a level of 0.05 (Benjamini and Hochberg, 1995). In all our tables, the symbol * indicates a p-value < 0.05, ** a p-value < 0.01, and *** a p-value < 0.001.

4.2 Quantitative analysis

After the training session, all users could perform lifting movement correctly. Table 1 shows the mean and standard deviation of kinematic (movement duration MD, arm displacement Disp) and kinetic (physical work) features. It should be highlighted that Disp for congruous conditions was approximately 1 m, but that, for incongruous conditions, it increased in relation with the object mass (reaching more than 2 m for 5 kg in the direct-weight condition; Table 1-displacement).

TABLE 1

TABLE 1. Kinematic features and physical work of weight lifting movement with different masses in virtual reality.

A significant change in MD was observed when subjects were asked to lift bottles of different weights (F (3, 1,598) = 10.578, p < 0.0001, ${\mathrm{\eta }}_p^2=0.019$). Different conditions also influenced MD (F (3, 1,598) = 71.208, p < 0.0001, ${\mathrm{\eta }}_p^2=0.118$). Furthermore, the interaction between conditions and weight significantly affected MD (F (91,589) = 7.317, p < 0.0001, ${\mathrm{\eta }}_p^2=0.04$). According to the Tukey post hoc analysis, MD significantly changed in relation to all multiple comparison conditions (p < 0.0001). A pair-wise comparison of B-H results did not reveal any false p-values. As a result of posthoc analysis, MD values for 0.616 and 1 kg were not statistically different (p > 0.05), as well as for 2 and 5 kg (p > 0.05), while MD values for all other comparisons were notably different (p < 0.05). In addition, the results of the B-H test confirm those obtained from the posthoc analysis.

Conditions significantly affected Disp (F (31,598) = 94.224, p < 0.0001, ${\mathrm{\eta }}_{\mathit{p}}^{\mathrm{2}}=\mathrm{0.15}$) as well as weight (F (31,598) = 44.998, p < 0.0001, ${\mathrm{\eta }}_{\mathit{p}}^{\mathrm{2}}=\mathrm{0.078}$). The interaction effect of condition and weight on disp was also significant (F (91,598) = 36.369, p < 0.0001, ${\mathrm{\eta }}_{\mathit{p}}^{\mathrm{2}}=\mathrm{0.170}$). Multiple-comparison Tukey post hoc analysis showed that there is a significant effect of different weights on Disp (p < 0.0001), except for 0.616 and 1 kg (p > 0.05). However, Tukey post hoc for all multiple comparisons of different conditions showed significant discrepancy between different conditions (p < 0.0001), except for speed-control and haptic (p > 0.05). B-H method confirmed all post hoc results (no false discovery).

Physical work significantly changed by different weights (F (3, 1,598) = 22.593, p < 0.0001, ${\mathrm{\eta }}_{\mathit{p}}^{\mathrm{2}}=\mathrm{0.041}$) as well as conditions (F (3, 1,598) = 118.036, p < 0.0001, ${\mathrm{\eta }}_{\mathit{p}}^{\mathrm{2}}=\mathrm{0.181}$). The interaction between condition and weight for work was also significant (F (9, 1,598) = 10.578, p < 0.0001, ${\mathrm{\eta }}_{\mathit{p}}^{\mathrm{2}}=\mathrm{0.09}$). Tukey’s HSD test for multiple comparison found that the mean value of work was significantly different between haptic and isomorphic (p < 0.0001, 95% C.I. = 41.65,58.00) as well as between haptic and direct-weight (p < 0.0001, 95% C.I. = 39.2055.26) and between haptic and speed-control (p < 0.0001, 95% C.I. = 40.08,56.28). Meanwhile, the mean value of work significantly changed between different weights. Post hoc HSD analysis demonstrated that there is a significant difference between 0.616 and 2 kg (p < 0.0001, 95% C.I. = −18.84,‒2.66), as well as between 0.616 and 5 kg (p < 0.0001, 95% C.I. = −30.65,‒14.49), 1 and 2 kg (p = 0.026, 95% C.I. = −16.93,‒0.75), 1 and 5 kg (p < 0.0001, 95% C.I. = −28.74,‒12.57), and 2 and 5 kg (p = 0.01, 95% C.I. = −19.91,‒3.72). The B-H method also confirmed the Tukey post hoc results and did not detect any false p-values for multiple comparison of conditions or weights.

4.2.1 Haptic condition

Figure 6 illustrates the mean value and standard deviation of the physical work performed by 20 subjects as they lifted real bottles in an upward movement with a real mass. Mean ± SD of physical work for this condition related to different weights released the tight connection between mass and physical work values (see Table 1-physical work, column Haptic). Because users were lifting real bottles here, the value of the physical work was significantly greater than that for the other conditions (see Table 1-physical work column Haptic). Results of one-way ANOVA for different weights (independent variable, four levels) in this condition on work values (dependent variable) showed that (F (3, 390) = 18.041, p < 0.0001). Tukey post hoc revealed significant differences for all paired comparisons (p < 0.05), except between weights of 0.616 and 1 kg (p > 0.05) and 1 and 2 kg (p > 0.05). B-H results confirm post hoc results.

FIGURE 6

FIGURE 6. Mean ± SD of work for 20 users for the reference (Haptic) condition.

4.2.2 Isomorphic condition

Figure 7 I shows the result of mean ± SD of work for 20 users in the isomorphic condition. The mean values of the physical work related to this condition for different masses are the same (see Table 1-physical work, column Isomorphic). As a result of the group analysis (one-way ANOVA) showed that, that the related physical activity did not vary significantly with different weights 0.616, 1, 2, and 5 kg as the virtual object’s mass increased F (3, 391) = 1.923, p = 0.126).

FIGURE 7

FIGURE 7. Mean ± SD of work for 20 users for the VR (non-haptic) conditions. From left to right: D = direct weight condition, I=Isomorphic condition, S = speed control condition.

4.2.3 Speed-control condition

In this condition, the subjects saw a visual conflict when their maximum velocity reached the threshold. For example, when subjects’ performance corresponded to lifting lighter virtual objects (0.616 or 1 kg) using the controller, they could easily adjust their velocity to achieve the authorized threshold. However, during the lifting of heavier objects (2 or 5 kg), the velocity of the virtual bottles usually was not aligned to the actual hand movement. Figure 7-S) illustrates the mean and standard deviation for physical work performed by twenty users under a speed control condition. As part of the speed control condition, a one-way ANOVA was conducted in order to compare the effects of virtual weight on work. ANOVA analysis confirms that virtual weights significantly affect physical activity [F (3, 396) = 9.204, p < 0.0001]. The Tukey post hoc analysis revealed a significant difference in work values between 0.616 and 2 kg (p 0.0001, 95% confidence interval = −2.76, −0.65), 0.616 and 5 kg [p 0.0001, 95% confidence interval = −2.75,−0.66)], 1 and 2 kg (p = 0.012, 95% confidence interval = −2.31, −0.19), and 1 and 5 kg (p = 0.011, 95% confidence interval = −2.32, 0.21). Based on the B-H results, no significant differences were observed between the work values associated with weights of 0.616 and 1 kg and 2 and 5 kg (p-value > 0.05).

4.2.4 Direct-weight condition

As Table 1- physical work (column direct-weight) shows, by increasing the weight of the virtual object, subjects’ physical work also increased (see Figure 7D). In addition, the results of one-way ANOVA revealed that virtual weight has a significant effect on physical work [F (3, 421) = 282.456, p < 0.0001]. Tukey post hoc analysis showed that different virtual weights significantly affect physical work (p < 0.05). B-H test results confirmed the Tukey post hoc results.

Table 3 demonstrates the results of a one-tailed paired sample t-test between pair conditions with the similar masses. Despite the small masses, these results suggest that the amount of physical effort users expend when lifting a virtual object is significantly different from real weightlifting. This analysis confirms that there is no significant difference between direct weights and isomorphics for the small mass (0.616 kg) (p-value > 0.05). In a similar manner, there are no differences between speed control and isomorphic conditions (p-value > 0.05). Statistical differences were evident for all masses (p-value< 0.05) except for 1 kg (p-value > 0.05).

Figure 8 indicates the 20 individual behaviors in the haptic and non-haptic conditions. By increasing the mass of the object, in the control condition, users follow a constant trend (also see Table 2, Isomorphic). However, in other conditions, users’ physical work follows an increasing trend.

FIGURE 8

FIGURE 8. Differences between different users’ behaviors in haptic and non-haptic conditions. (A) Individual values (dots) and mean values (bar plot) of work in different conditions in VR for different masses. (B) Individual and mean values of “physical work” (bar plot) in reference conditions (haptic). Data are shown as mean ± SD. Gary lines present users’ behavior trends for different masses.

TABLE 2

TABLE 2. One-way ANOVA results for the physical work related to different weight in each specific condition. Asterisks show the significances.

TABLE 3

TABLE 3. Result of t-test analysis for physical work between different conditions and same masses (D: direct-weight, H: haptic, S: speed-control, I: isomorphic).

Figure 8A shows different individual behaviors for the non-haptic conditions. In the direct-weight condition (D), by increasing the heaviness of the virtual bottle, users follow the same trend as the haptic condition (see Figure 8B). It is therefore evident that in the direct weight condition, by increasing the mass of the object, the physical work of the users is augmented (work performed by approximately 95% of users), in a similar manner to the haptic condition (75% of users). The increase in mass also affected the work value of each user in the speed-control condition (see Figure 8A,S); however, this fluctuation between 2 and 5 kg was not substantial (T-test, p-value = 0.05), and only 30% of the users’ physical work value increased. The isomorphic condition (see Figure 8A,I) shows that work values did not change statistically across masses (see Table 2), and only 5% of users could determine the object’s weight.

4.3 Qualitative analysis

4.3.1 Borg CR10

After each condition, users were asked to complete a Borg CR10 questionnaire to measure their level of fatigue. To assess the level of fatigue among 20 subjects, we calculated the mean and standard deviation of the fatigue levels (Figure 9). In contrast to the other conditions, users did not experience tiredness in their hands when they were in the isomorphic condition. The mean values of the fatigue questionnaire for the direct-weight and haptic conditions are 3.65 ± 2.4 and 3.65 ± 2.66, respectively. It is noteworthy that although the users reported the same level of fatigue in their hands during the haptic and direct-weight conditions, the level of tiredness in these conditions increased in comparison with the isomorphic condition. According to the one-way ANOVA test, subjects’ sense of tiredness was significantly affected by the conditions under which they were tested [F (3, 77) = 6.600, p < 0.0005]. Based on the post hoc analysis, there are no significant differences between the haptic and direct-weight conditions, nor between the haptic and speed-control conditions (ps > 0.05). In addition, the B-H results support the conclusions reached in the post hoc analysis.

FIGURE 9

FIGURE 9. Results of the modified Borg CR10 questionnaire: mean and standard deviation of fatigue for 20 users after performing each condition.

4.3.2 Presence questionnaire

Another questionnaire we used to subjectively measure the sense of presence in our VE was a presence questionnaire (Witmer and Singer, 1998). After performing each condition, users were asked to fill out this questionnaire and explain how they felt.

A plot of the mean values of quality of the interface, possibility to examine, self-evaluation of performance, haptics, realism, and possibility to act is shown in Figure 10. It is evident from this figure that users had similar feelings in all conditions. Table 4 presents the results of an ANOVA group analysis for each feature of this questionnaire among all users under different conditions. Under different experimental conditions, there was no significant difference in presence features (see Table 4, p-value > 0.05).

FIGURE 10

FIGURE 10. Presence questionnaire: Mean of responses of 20 users to measure the different features of presence questionnaire, completed by users after each condition.

TABLE 4

TABLE 4. Results of ANOVA test for presence questionnaire features between different conditions (H: Haptic, I: isomorphic, S: speed-control, D: direct-weight).

5 Discussion

In this study, we conducted an experiment in the context of lifting an object in virtual reality. Four conditions were evaluated, two congruous and two incongruous, in terms of perceived and performed movements. We used a haptic as the reference condition. Finally, we assessed the perception of the effort to lift the object for each condition through objective measurements (physical work) and subjective measurements (Borg CR10 questionnaire and presence questionnaire).

As expected and as a consequence of our pseudo-haptic feedback approaches, the actual hand displacement feature (disp; see Table 1-displacement) showed that the value of the disp is approximately 1 m in the congruous conditions. While in incongruous conditions (speed-control and direct-weight), by increasing the weight of the virtual object, the value of the disp feature increases.

Two-way ANOVA results also showed that the displacement feature remained consistent during bottle lifting with different weights in different conditions. However, when subjects lifted 0.616 and 1 kg, their hand displacement remained consistent, while heavier bottles led to changes in displacements. Therefore, according to the displacement feature, the weight of one kg could be a threshold weight that participants can lift both in VR and real environments without noticing any difference in their hand displacements.

Regarding movement duration, the result of the two-way ANOVA showed that it was not stable for congruous and incongruous conditions and different weights. This inconsistency of the movement duration showed that users tried to compensate for the weight of the object by incrementing their actual hand movement distance. Therefore, logically, this displacement increase leads to an increased movement duration.

The results of the fatigue questionnaire showed that users sensed the same amount of fatigue in their hands when they were lifting real objects in haptic conditions and when they lifted virtual bottles in the direct-weight condition. Similarly, the results of the presence questionnaire showed that users in non-haptic conditions had the same sense of presence that they had in haptic conditions. More generally, Table 4 confirms that there were no significant differences between users’ feelings of presence in different conditions. These results demonstrate that users were immersed in the VE and that we succeeded in inducing the same amount of tiredness in haptic and direct-weight conditions.

Despite the large differences in physical work between haptic and isomorphic conditions, pseudo-haptic approaches strongly impact individual physical work (see Figure 6; Figure 7). Our formulation did not take into account the mass of the subjects’ arms, which may explain the large discrepancy. The formula for work (Eq. 10) implies that work is directly related to the weight of an object. However, Figure 8 indicates that for some individuals, the value of work decreases when the weights of the objects are increased from 2 to 5 kg in the haptic condition. Meanwhile, all users, with the exception of one, appear to be able to increase the value of their work as a result of increasing the weight of the virtual objects. In VR, users modify their lifting behaviors, and our pseudo-haptic methods affect their movement patterns. This cannot be accidental; comparing direct-weight to isomorphic conditions, users did not recognize the weights of the virtual objects in the isomorphic condition since there was no pseudo-haptic feedback. Therefore, they performed the same hand movements for all weights in the isomorphic condition.

Likewise, in speed-control conditions, users are able to observe the effects of visuomotor conflict on their hand movements. Comparatively to the isomorphic condition, visual feedback has a significant impact on the work’s value. In speed control, however, the virtual bottle’s velocity changes as a function of the weight of the bottle. For 0.616 kg, there is no statistically significant difference between the isomorphic condition and the speed-control condition, whereas for heavier weights (1, 2, and 5 kg), there are significant differences. In other words, the integration of multi-sensory conflict affects the acceleration and displacement of the users’ hands, resulting in them performing more actions on the controller.

By comparing the findings between the speed-control and direct-weight conditions, we discovered that the direct-weight condition caused subjects to increase their weights to 2 and 5 kg more effectively (post hoc-Tukey, p-value 0.05). While subjects perceived the difference between 2 and 5 kg in the speed-control condition (post hoc Tukey, p-value > 0.05).

Our methods (direct-weight and speed control) have been shown to simulate fatigue similar to haptic conditions in the hands of users (Borg CR10). Therefore, our methods were successful in terms of enhancing the user’s sense of presence.

We identify a number of benefits for VR designers who are faced with the challenge of providing virtual object weight to the user of a VR hand controller. This study’s primary strength is direct interaction with virtual objects using a VR controller and trackers. Secondly, we induced heavier weights that are close to the weight of objects that people encounter in their daily lives, while previous studies focused on objects that weigh less than 500 g (Samad, et al., 2019). A pseudo-haptic model can successfully induce these weights in VR participants without any additional haptic devices, when the weight of the virtual object is equal to or less than 1 kg. However, for heavier virtual weights such as (2 and 5 kg), subjects may experience the same fatigue in their hands as in the haptic condition. To avoid discrepancies between visual and kinaesthetic cues, extra haptic devices should be used. Doing so ensures that the subject will not be affected by the VR environment in terms of their sense of presence.

Consequently, these approaches could be beneficial in other VR fields and decrease the reliance of VR on hardware devices. Weight perception can be used to interact more naturally with virtual objects rather than limiting the interaction with the VE. The other contribution of our work is the use of the physical work formula as an objective measurement evaluator based on kinematic data. As a result, subjective judgment is not required. There is generally no real-world testing of haptic interfaces in the evaluation process, which results in inaccurate weight perception assessment results (Lim, et al., 2021). We conducted different lifting conditions with both real objects and virtual bottles to clearly evaluate the object’s weight perception.

6 Conclusion

Inducing the a feeling of weight of virtual objects to VR users is a difficult task due to lack of force feedback cues. This research focused on simple force feedback simulation approaches, through visual cues, instead of using expensive and large hardware and physical interfaces. By utilizing the concept of pseudo-haptic feedback, we designed two different approaches based on the C/D ratio concept and aimed to evaluate the effect of each approach on human behavior lifting in VR without extra real weight and compared it with the effect of real object lifting. To evaluate our findings, we benefited from the concept of physical work and different questionnaires (fatigue, presence). According to our findings, the modified C/D ratio method based on distance control can induce virtual object heaviness (and in a better way than velocity control), with object lifting behavior close to real object lifting one. This finding could be interesting for VR developers who want to develop more reliable VE with such tasks.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study.

Author contributions

MM, PR, CG, JP, and FM contributed to conception of the study. MM, PR, CG, and FM contributed to design the protocols and conditions of the study. PR developed the virtual reality application and the data recording system. MM organized the database and performed the statistical analysis. MM and PR wrote the first draft of the manuscript. CG and FM wrote sections of the manuscript. MM, PR, CG, and FM contributed to manuscript revision, read, and approved the submitted version.

Acknowledgments

This work was conducted using the equipment of the technological platform of the Lispen laboratory in the framework of the collaborative project Continuum. The authors want to thank Jean-Luc Martinez who proposed the idea of pseudo-haptic interaction for the weighting task.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Aman, J., Lu, C.-h., and Konczak, J. (2010). The integration of vision and haptic sensing: A computational & neural perspective. Integration 96, 98. Minneapolis, United States: University of Minnesota.

Google Scholar

Andujar, C., and Argelaguet, F. (2007). “Virtual pads: Decoupling motor space and visual space for flexible manipulation of 2d windows within ves,” in Proceeding of the 2007 IEEE Symposium on 3D User Interfaces, Charlotte, NC, USA, March 2007 (IEEE). s.l., s.n.

CrossRef Full Text | Google Scholar

Argelaguet, F., and Andujar, C. (2013). A survey of 3D object selection techniques for virtual environments. Comput. Graph. 37, 121–136. doi:10.1016/j.cag.2012.12.003

CrossRef Full Text | Google Scholar

Benjamini, Y., and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B Methodol. 57, 289–300. doi:10.1111/j.2517-6161.1995.tb02031.x

CrossRef Full Text | Google Scholar

Borg, G. (1990). Psychophysical scaling with applications in physical work and the perception of exertion. Scand. J. Work, Environ. Health 16, 55–58. doi:10.5271/sjweh.1815

PubMed Abstract | CrossRef Full Text | Google Scholar

Bowman, D. A., and Hodges, L. F. (1999). Formalizing the design, evaluation, and application of interaction techniques for immersive virtual environments. J. Vis. Lang. Comput. 10, 37–53. doi:10.1006/jvlc.1998.0111

CrossRef Full Text | Google Scholar

Bowman, D. A., Johnson, D. B., and Hodges, L. F. (2001). Testbed evaluation of virtual environment interaction techniques. Presence 10, 75–95. doi:10.1162/105474601750182333

CrossRef Full Text | Google Scholar

Burdea, G. C. (1996). Force and touch feedback for virtual reality. New York: John Wiley & Sons. s.l.

Google Scholar

Burdea, G. C. (1999). “Haptic feedback for virtual reality,” in Proceedings of international workshop on virtual prototyping (Laval, France), 17–29. s.l., s.n.

Google Scholar

Choi, I., Culbertson, H., Miller, M. R., Olwal, A., and Follmer, S. (2017). “Grabity: A wearable haptic interface for simulating weight and grasping in virtual reality,” in UIST '17: Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, October 2017 (New York, NY, United States: Association for Computing Machinery), 119–130. s.l., s.n.

Google Scholar

De Tinguy, X., Pacchierotti, C., Marchal, M., and Lécuyer, A. (2018). “Enhancing the stiffness perception of tangible objects in mixed reality using wearable haptics,” in Proceedings of the 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Tuebingen/Reutlingen, Germany, March 2018 (IEEE), 81–90. s.l., s.

CrossRef Full Text | Google Scholar

Dominjon, L., Lecuyer, A., Burkhardt, J. M., Richard, P., and Richir, S. (2005). “Influence of control/display ratio on the perception of mass of manipulated objects in virtual environments,” in Proceedings of the IEEE Proceedings. VR 2005. Virtual Reality, 2005, Bonn, Germany, March 2005 (IEEE), 19–25. s.l., s.n.

Google Scholar

Ernst, M. O., and Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429–433. doi:10.1038/415429a

PubMed Abstract | CrossRef Full Text | Google Scholar

Frees, S., Kessler, G. D., and Kay, E. (2007). PRISM interaction for enhancing control in immersive virtual environments. ACM Trans. Computer-Human Interact. (TOCHI) 14, 2. –es. doi:10.1145/1229855.1229857

CrossRef Full Text | Google Scholar

Gibbs, J. K., Gillies, M., and Pan, X. (2022). A comparison of the effects of haptic and visual feedback on presence in virtual reality. Int. J. Human-Computer Stud. 157, 102717. doi:10.1016/j.ijhcs.2021.102717

CrossRef Full Text | Google Scholar

Gillan, D. J., Holden, K., Adam, S., Rudisill, M., and Magee, L. (1990). “How does Fitts' law fit pointing and dragging?,” in CHI '90: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, March 1990, 227–234. s.l., s.n.

CrossRef Full Text | Google Scholar

Hagadorn, K. T. (2004). The effects of object weight and three-dimensional movement on human movement time and Fitts' law. Thesis. Rochester, NY, United States: Rochester Institute of Technology. Available at: https://scholarworks.rit.edu/theses/7719

Google Scholar

Jauregui, D. A. G., Argelaguet, F., Olivier, A. H., Marchal, M., Multon, F., and Lecuyer, A. (2014). Toward “pseudo-haptic avatars”: Modifying the visual animation of self-avatar can simulate the perception of weight lifting. IEEE Trans. Vis. Comput. Graph. 20, 654–661. doi:10.1109/tvcg.2014.45

PubMed Abstract | CrossRef Full Text | Google Scholar

Lécuyer, A., Coquillart, S., Kheddar, A., Richard, P., and Coiffet, P. (2000). “Pseudo-haptic feedback: Can isometric input devices simulate force feedback?,” in Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048), New Brunswick, NJ, USA, March 2000 (IEEE), 83–90. s.l., s.n.

Google Scholar

Lécuyer, A., Burkhardt, J.-M., and Etienne, L. (2004). “Feeling bumps and holes without a haptic interface: The perception of pseudo-haptic textures,” in CHI '04: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, April 2004 (New York, NY, United States: Association for Computing Machinery), 239–246. s.l., s.n.

Google Scholar

Lécuyer, A. (2009). Simulating haptic feedback using vision: A survey of research and applications of pseudo-haptic feedback. Presence Teleoperators Virtual Environ. 18, 39–53. doi:10.1162/pres.18.1.39

CrossRef Full Text | Google Scholar

Lee, J., Kim, J.-I., and Kim, H. (2019). “Force arrow 2: A novel pseudo-haptic interface for weight perception in lifting virtual objects,” in Proceeding of the 2019 IEEE International Conference on Big Data and Smart Computing (BigComp), Kyoto, Japan, February 2019 (IEEE), 1–8. s.l., s.n.

CrossRef Full Text | Google Scholar

Lim, W. N., Yap, K. M., Lee, Y., Wee, C., and Yen, C. C. (2021). A systematic review of weight perception in virtual reality: Challenges and road ahead. IEEE Access 9, 163253–163283. doi:10.1109/ACCESS.2021.3131525

CrossRef Full Text | Google Scholar

MacKenzie, I. S. (1993). Fitts' law as a performance model in human-computer interaction. Doctoral dissertation. Toronto, Ontario, Canada: University of Toronto.

Google Scholar

Maehigashi, A., Sasada, A., Matsumuro, M., Shibata, F., Kimura, A., and Niida, S. (2021). “Virtual weight illusion: Weight perception of virtual objects using weight illusions,” in Conference: CHI '21: CHI Conference on Human Factors in Computing Systems, May 2021 (New York, NY, United States: Association for Computing Machinery), 1–6. s.l., s.n.

CrossRef Full Text | Google Scholar

Massie, T. H., and Salisbury, J. K. (1994). “The phantom haptic interface: A device for probing virtual objects,” in Proceedings of the ASME Winter Annual Meeting, Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Chicago, IL, Nov. 1994 (Chicago: ACM), 295–300. s.l., s.n.

Google Scholar

Nakakoji, K., Yamamoto, Y., and Koike, Y. (2010). “Toward principles for visual interaction design for communicating weight by using pseudo-haptic feedback,” in The interaction design conference, July 2010 (Swindon, United Kingdom: BCS Learning & Development Ltd.), 1–6. s.l., s.n.

CrossRef Full Text | Google Scholar

Nakakoji, K., Yamamoto, Y., Matsubara, N., and Koike, Y. (2011). “Tciex: An environment for designing and experiencing a variety of visuo-haptic sensory conflicts,” in Proceedings of the 3rd IEEE VR2011 Workshop on Perceptual Illusions in Virtual Environments (PIVE 2011), Singapore, March 2011, 23–26.

Google Scholar

Nguyen, H., and Bednarz, T. (2020). “User experience in collaborative extended reality: Overview study,” in Proceedings of the 17th EuroVR International Conference. Valencia, Spain: Springer International Publishing. November 25–27 2020 (ACM), 41–70.

CrossRef Full Text | Google Scholar

Nisar, S., Martinez, M. O., Endo, T., Matsuno, F., and Okamura, A. M. (2018). Effects of different hand-grounding locations on haptic performance with a wearable kinesthetic haptic device. IEEE Robotics Automation Lett. 4, 351–358. doi:10.1109/lra.2018.2890198

CrossRef Full Text | Google Scholar

Pacchierotti, C., Sinclair, S., Solazzi, M., Frisoli, A., Hayward, V., and Prattichizzo, D. (2017). Wearable haptic systems for the fingertip and the hand: Taxonomy, review, and perspectives. IEEE Trans. Haptics 10, 580–600. doi:10.1109/toh.2017.2689006

PubMed Abstract | CrossRef Full Text | Google Scholar

Palmerius, K. L., Johansson, D., Höst, G., and Schönborn, K. (2014). “An analysis of the influence of a pseudo-haptic cue on the haptic perception of weight,” in Proceedings of the International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, July 2010 (Berlin, Heidelberg: Springer), 117–125. s.l., s.n.

CrossRef Full Text | Google Scholar

Poupyrev, I., and Ichikawa, T. (1999). Manipulating objects in virtual worlds: Categorization and empirical evaluation of interaction techniques. J. Vis. Lang. Comput. 10, 19–35. doi:10.1006/jvlc.1998.0112

CrossRef Full Text | Google Scholar

Poupyrev, I., Billinghurst, M., Weghorst, S., and Ichikawa, T. (1996). “The go-go interaction technique: Non-linear mapping for direct manipulation in VR,” in UIST '96: Proceedings of the 9th annual ACM symposium on User interface software and technology, Nov. 1996 (New York, NY, United States: Association for Computing Machinery), 79–80. s.l., s.n.

Google Scholar

Poupyrev, I., Weghorst, S., Billinghurst, M., and Ichikawa, T. (1998). A study of techniques for selecting and positioning objects in immersive VEs: Effects of distance, size, and visual feedback. s.l., s.n.

Google Scholar

Ramsamy, P., Haffegee, A., Jamieson, R., and Alexandrov, V. (2006). “Using haptics to improve immersion in virtual environments,” in ICCS'06: Proceedings of the 6th international conference on Computational Science - Volume Part II, May 2006 (Berlin, Heidelberg: Springer), 603–609. s.l., s.n.

CrossRef Full Text | Google Scholar

Rietzler, M., Geiselhart, F., Gugenheimer, J., and Rukzio, E. (2018). Breaking the tracking: Enabling weight perception using perceivable tracking offsets in CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, April 2018 (New York, NY, United States: Association for Computing Machinery), 1–12. s.l., s.n.

Google Scholar

Rock, I., and Victor, J. (1964). Vision and touch: An experimentally created conflict between the two senses. Science 143, 594–596. doi:10.1126/science.143.3606.594

PubMed Abstract | CrossRef Full Text | Google Scholar

Rosa, N., Hürst, W., Vos, W., and Werkhoven, P. (2015). “The Influence of visual cues on passive tactile sensations in a multimodal immersive virtual environment,” in ICMI '15: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, Nov. 2015 (New York, NY, United States: Association for Computing Machinery), 327–334. s.l., s.n.

CrossRef Full Text | Google Scholar

Runeson, S., and Frykholm, G. (1981). Visual perception of lifted weight. J. Exp. Psychol. Hum. Percept. Perform. 7, 733–740. doi:10.1037/0096-1523.7.4.733

PubMed Abstract | CrossRef Full Text | Google Scholar

Samad, M., Gatti, E., Hermes, A., Benko, H., and Parise, C. (2019). “Pseudo-haptic weight: Changing the perceived weight of virtual objects by manipulating control-display ratio,” in CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, May 2019 (New York, NY, United States: Association for Computing Machinery), 1–13. s.l., s.n.

Google Scholar

Sciutti, A., Squeri, V., Gori, M., Masia, L., Sandini, G., and Konczak, J. (2010). Predicted sensory feedback derived from motor commands does not improve haptic sensitivity. Exp. Brain Res. 200, 259–267. doi:10.1007/s00221-009-1996-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Ujitoko, Y., and Ban, Y. (2021). Survey of pseudo-haptics: Haptic feedback design and application proposals. IEEE Trans. Haptics 14, 699–711. doi:10.1109/toh.2021.3077619

PubMed Abstract | CrossRef Full Text | Google Scholar

Witmer, B. G., and Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence 7, 225–240. doi:10.1162/105474698565686

CrossRef Full Text | Google Scholar

Yu, R., and Bowman, D. A. (2020). Pseudo-haptic display of mass and mass distribution during object rotation in virtual reality. IEEE Trans. Vis. Comput. Graph. 26, 2094–2103. doi:10.1109/tvcg.2020.2973056

PubMed Abstract | CrossRef Full Text | Google Scholar

Zenner, A., and Krüger, A. (2017). Shifty: A weight-shifting dynamic passive haptic proxy to enhance object perception in virtual reality. IEEE Trans. Vis. Comput. Graph. 23, 1285–1294. doi:10.1109/tvcg.2017.2656978

PubMed Abstract | CrossRef Full Text | Google Scholar