Discrete Cutaneous Feedback for Reducing Dimensions of Wearable Haptic Devices

In this article, we explore alternative cutaneous haptic feedback for rendering modulation of the grasping force. The aim of the study was to reduce power requirements and in turn dimensions of the actuators, in wearable devices applied to virtual or teleoperated manipulation. This is critical in certain rehabilitation or training scenarios where haptics should not interfere with dexterity of the user. In the study, we experimented discrete, pulsed cutaneous force feedback and compared it with conventional continuous proportional feedback, in a virtual pick and place task. We made use of wearable thimbles based on voice coil actuators in order to provide high-quality, low-noise haptic feedback to the participants. The evaluation was performed on the basis of both objective measurements of task performance (measured virtual forces and correct ratio) and a questionnaire evaluating participants’ preferences for the different feedback conditions. On the basis of the obtained results, in the article, we discuss the possibility of providing high-frequency, discretized cutaneous feedback only, driven by modulation of the grasping force. The opportunity is to reduce volume and mass of the actuators and also to consider alternative design solutions, due to the different requirements in terms of static and high-frequency components of the output force.

Introduction

Haptic rendering of physical interaction in teleoperated or virtual manipulation tasks is a challenging and extensively explored research field; in manipulation tasks, haptic perception is a fundamental sensory pathway for task execution. On the other hand, development of haptic devices for the fingertips has to comply with strict constraints in terms of wearability, lightweight, interfinger interferences, and quality of the provided feedback. Concerning the design of wearable haptic devices for the fingertip segment, the literature includes a variety of proposed designs for rendering specific haptic cues, such as vibratory (Solazzi et al., 2010), contact orientation (Chinello et al., 2015), contact force (Leonardis et al., 2015), area of contact Fani et al. (2017), thermal (Gallo et al., 2015), or a combination of the aforementioned feedback (Wang et al., 2019; Gabardi et al., 2018). An extensive review of portable and wearable haptic devices for the fingertips can be found in Pacchierotti et al., (2017), in which also taxonomy for such devices is introduced. Recent research applications of haptic feedback include rehabilitation systems based on virtual exercises Gutiérrez et al. (2021), virtual serious games (Bortone et al., 2017), and robotic teleoperation (Klamt et al., 2020).

To replicate the correct physical interaction that occurs when the fingertip touches a virtual object is a challenging objective (Caldwell et al., 1997), and concerning fingertip haptic devices, practical requirements such as wearability and portability impose limits to the feedback the device can render or a trade-off between quality of the feedback and wearability of the device. Limitations can be, for instance, in the bandwidth and maximum output force amplitude, limits in the range of motion for shape-rendering thimbles, and heat flux intensity and dynamics for thermal devices.

About force rendering, the range of forces usually exchanged between the fingertips and a manipulated object during a natural interaction is usually too wide to be correctly reproduced by portable and wearable haptic devices. Moreover, since wearable haptic thimbles are grounded at the user fingers, such interfaces are able to provide the user with cutaneous cues only, without kinesthetic feedback. It means that in a virtual grasping task, perception of pressing forces can be delivered at the fingerpad by the actuated thimble, but the rendered force does not constrain the whole finger movement at the contact surface, as it would happen in a real physical grasping. A wearable device is only able to apply net forces between the actuated segment (i.e., the fingerpad) and segment the device is grounded at (i.e., the finger dorsum). Although it is not possible to reproduce the real contact constraints at the user’s fingertips, it is still possible to provide rich and informative feedback useful to accomplish virtual (Leonardis et al., 2017), augmented (Maisto et al., 2017), or teleoperated (Pacchierotti et al., 2015) manipulation tasks. Two techniques have been proposed to use cutaneous feedback only to convey haptic information to the user: the sensory subtraction (Prattichizzo et al., 2012) and the sensory substitution (Schorr et al., 2013).

Haptic feedback related to modulation of the contact force is widely explored in teleoperation and haptic interaction research and is also relevant in fine-manipulation; modulation of the right amount of normal force is critical to overcome slippage and at the same time prevent damage to the object or to the robotic fingers. It has been shown that in manipulation tasks, such as grasping and lifting an object, healthy individuals can accurately modulate grasping forces using information obtained from mechanoreceptors in the fingertips (Westling and Johansson, 1987). In particular, in natural execution of a grasping task, fingertip forces are held just above the level of static friction required to avoid slippage (Cole and Abbs, 1988). In case of virtual environments, where the feedback modalities are limited, the safety margin in terms of forces required to grasp and lift an object is increased (Bergamasco et al., 2006). Perception of force modulation can also carry information about geometrical properties of the explored surface, such as bumps or holes (Robles-De-La-Torre and Hayward, 2001).

The simplest haptic feedback strategy to render modulated grasping force consists in a linear function between the indentation of the finger through the virtual contact surface and the output force applied by the haptic device to the user’s fingerpad (Basdogan and Srinivasan, 2002). More complex models take into account also velocity and acceleration of a dynamic proxy (Niemeyer and Mitra, 2005). Furthermore, an interesting method for generating appropriate transients during virtual impacts is described in Kuchenbecker et al. (2006).

The aforementioned continuous force feedback strategies need to render a static component of the output force once the object is grasped, proportional to the grasping force required to hold the manipulated object. In terms of device design, it results, in general, in implementation of bigger actuators. Considering electromagnetic actuators (i.e., DC micromotors and voice coils), widely used in wearable haptic devices, the maximum continuous output force is limited by heat dissipation and maximum temperature reached in the windings. A bigger actuator is capable of higher output forces given the same maximum working temperature. Implementation of a mechanical reduction is another design solution that can be considered to obtain higher output forces. On the other hand, it involves several considerations and limits, especially in the wearable haptics field. Gear reduction is a conventional reduction method that can be implemented in miniaturized, lightweight servomotors, already proposed in the design of fingertip haptic devices ((Leonardis et al., 2017), (Chinello et al., 2015), (Pacchierotti et al., 2016)). However, gear reduction introduces noise and vibrations in the rendered signals, hence diminishing quality of the feedback. Also, it severely limits the output bandwidth of the rendered signals due to effects of backlash and friction. We have also proposed alternative reduction methods for fingertip haptic devices, such as screw reduction (Leonardis et al., 2020) and twist actuators (Leonardis et al., 2021), in order to limit or eliminate noise added by the reduction mechanism to the haptic rendering. The drawback remains a limited output bandwidth, friction, and non-backdrivability of the actuator, if compared with direct drive actuators (rotary motors or linear voice coils), which provide higher quality feedback at the cost of heavier and more cumbersome devices (Gabardi et al., 2016).

On the other hand, the rendering of the high-intensity, static force components might not be the most useful and informative part of the provided haptic feedback. In prosthetics, the use of discrete haptic feedback to inform the user of contact events, called DESC (Discrete Event-driven Sensory feedback Control), has been shown effective to improve control of grasp (Cipriani et al., 2014). Here, the haptic feedback is event-driven and carries information related to, i.e., object contact, liftoff, replace, and release events. The discrete nature of the feedback makes it suitable for effective implementation in prosthetics applications by means of vibrotactile motors; they can be integrated in the robotic prosthesis itself or located at a more proximal body segment, such as the arm or the forearm (Clemente et al., 2016). DESC feedback, however, does not carry information of force modulation during grasping. In Cappello et al. (2020), a study comparing DESC feedback with supplementary continuous force feedback is presented. Removal of the slow components of the continuous feedback (transient feedback) is envisaged in the study as a possibility to convey similar perceptual information to the user, diminishing required power of the haptic actuators at the same time.

In the present study, we propose the use of a discrete haptic feedback method also during the grasping loading phase, after the contact transition, in order to allow a closed-loop modulation of the grasping force, although without the use of continuous force components. Modulation of the grasping force is rendered here as a sequence of indentation-triggered discrete pulses.

By substituting static forces applied to the fingers with proper pulses associated to changes in the applied virtual grasping force, the user is able to keep stability of the grasping and capability of force control. By using the pulse-based force feedback strategy, the power required by the actuation system is expected to be significantly reduced. Regarding the actuator, while continuous output force components have to be limited within the nominal force value of the actuators (which in turn depends on its size and volume), isolated force peaks of much higher amplitude can be sustained without overcoming the maximum allowed temperature. Regarding perception, it has to be considered that part of the fingerpad mechanoreceptors are highly sensitive to fast dynamic signals (Bensmaıa and Hollins, 2000), up to the point the tactile sensory channel mixes with the auditory channel (Yau et al., 2009). Perception of such high-frequency haptic signals becomes extremely informative during virtual exploration (Wiertlewski et al., 2011) and telemanipulation (Kuchenbecker et al., 2010). Hence, we expect cues of the pulse-based feedback can be neatly perceived by the user even if the total energy of the signal is less than the conventional continuous feedback.

Pulse-based haptic feedback have been proposed in other scenarios: in Kurihara et al. (2013), a similar feedback was proposed to provide information about the angular position of a pedal control device in a driving simulator, and in Strohmeier et al. (2018) for rendering spatial information in a hand-held device, through an inertial-mass actuator. In Stepp and Matsuoka (2011), an interesting experiment compared two different modulation methods (amplitude or rate modulation of discrete vibration bursts) to render force interaction at the fingerpad. However, the referenced works lacked an experimental comparison of discrete or vibrotactile stimuli with the conventional continuous feedback, which is the closest one to the natural perception of grasping force modulation. Also, in the present study, we experimentally compare different feedback in a virtual pick-and-place task, allowing to better evaluate feedback effectiveness in a general and familiar manipulation task.

Summarizing, in this article, we propose and evaluate the use of discrete haptic feedback to render modulation of grasping force in virtual manipulation tasks (Figure 1). The aim was to reduce power requirements of the actuators and in turn dimensions while providing informative haptic feedback. The improved wearability, especially at the fingertip level, is considered critical in certain training or rehabilitation virtual task in which the addition of haptic devices should not interfere with user’s dexterity. To evaluate effectiveness of the proposed feedback, a pick-and-place virtual task has been developed and experimented in different force feedback conditions. Objective metrics and a perception questionnaire were used to investigate which feedback provided better performance. The article is organized as follows: in Section 2, the adopted feedback strategies are described. Then, the experimental setup is presented in Section 3.3. Obtained results are presented in Section 4 and discussed in Section 5. Finally, conclusions are reported in Section 6.

FIGURE 1

FIGURE 1. Hand of a subject, wearing the haptic interfaces and approaching a virtual cube to accomplish the manipulation task.

Force Feedback Strategies

The use of high-frequency haptic stimuli for modulating the perception of interaction with a physical, rigid surface has been proposed in Kildal (2010). In the study, the illusion of compliant surfaces was elicited by providing high-frequency vibrotactile transients at discrete thresholds of the increasing normal force. In Visell et al. (2014), authors showed the perceptual integration of vibrotactile and force–displacement cues could modulate the perceived stiffness during one finger interaction. In the study, the vibration noise was added to force–displacement feedback, and noise amplitude was modulated by the rate of increasing loading force.

In this study, we developed a conceptually similar force feedback strategy, based on discrete haptic cues triggered by the exerted virtual grasping force. The purpose was to enhance perception and control of the virtual grasping force and to eventually substitute the continuous force component. Including the conventional continuous force–displacement strategy, we developed three different feedback conditions to be experimentally evaluated and compared, depicted in Figure 2 and analytically described as:

FIGURE 2

FIGURE 2. Three haptic feedback strategies evaluated in the experiments. Dotted gray lines represent hysteresis applied to the force pulse thresholds.

• Feedback 1: Proportional (P)

$F=\left\{\begin{array}{cc}0\hfill & \text{\hspace{0.17em}for}\delta <0\hfill \\ k_c\delta \hfill & \text{\hspace{0.17em}for}\delta \ge 0\hfill \end{array}\right.(1)$

where δ is the indentation depth of the finger proxy in the virtual surface, and k_c is the constant ratio between the indentation distance and the force feedback value. k_c was set to 50 mN/mm.

• Feedback 2: Discrete (D)

$F=\left\{\begin{array}{cc}0\hfill & \text{\hspace{0.17em}for}\delta \le 0\hfill \\ k_b\hfill & \text{\hspace{0.17em}for}\delta \ge 0\text{and}\dot{\delta }\le 0\hfill \\ k_b+k_{p}II{I}_T\hfill & \text{\hspace{0.17em}for}\delta \ge 0\text{and}\dot{\delta }\ge 0\hfill \end{array}\right.(2)$

where $II{I}_T={\sum }_{k=0}^n\mathrm{\Delta }(\delta -kT)$ can be described as a Dirac comb (or an impulse train) obtained as distributions of Dirac delta (Δ) functions. T = 0.8 mm is the spatial distance between successive pulses. The k_b force contribution is very low, and it is used to avoid the separation that occurs between the finger and the plate when δ ≥ 0. k_b was set to 50 mN, while k_p was set to 150 mN.

Each pulse lasted for 1 ms. Moreover, 0.2 mm of hysteresis has been applied to each threshold in order to avoid oscillations of the proxy position across the threshold. According to Equation 2, the pulses are rendered only when a quantized positive increment of the indentation depth occurs. No feedback was provided in the releasing direction, in order to make participants to unambiguously associate the feedback to a clear motor action (increment of the grasping force). In order to set a consistent value for the spacing between pulses, we conducted a brief exploratory investigation involving three subjects (male, age 29–35). The experiment involved a simplified setup (fingertip haptic devices and optical tracking system only) and proposed the subjects to precisely close the fingers between two minimum and maximum distance threshold (34: mm and 37 mm, respectively, with feedback starting below 40 mm distance). The task result was shown to the subjects after each repetition. We explored 0.4, 0.8 , and 1.6 mm spacing between pulses. The averaged results in terms of variability between repetitions showed similar results for the 0.4 and 0.8 mm spacing, (respectively, 1.2 and 1.0 mm standard deviation) with a noticeably higher variability for the 1.6 mm spacing (2.5 mm standard deviation).

• Feedback 3: Proportional + Discrete

$F=\left\{\begin{array}{cc}0\hfill & \text{\hspace{0.17em}for}\delta \le 0\hfill \\ k_c\delta \hfill & \text{\hspace{0.17em}for}\delta \ge 0\text{and}\dot{\delta }\le 0\hfill \\ k_c\delta +k_{p}II{I}_T\hfill & \text{\hspace{0.17em}for}\delta \ge 0\text{and}\dot{\delta }\ge 0\hfill \end{array}\right.(3)$

Considering negligible the k_b force contribution of feedback 2, it holds the relation PD = P + D.

Experimental Methods

The Virtual Task

The experiment was based on a virtual pick-and-place manipulation task. The virtual environment included a virtual desk with two fixed platforms. When a pick-and-place trial started, a cube appeared on top of the left platform. Each trial consisted in accurately moving the cube from the left platform to the right one. Each trial could end with the cube correctly placed on top of the right platform or with a failed trial; the cube could fall on the desk, due to grasping force below simulated static friction, or the cube could break when the grasping force overcame a fixed threshold.

Finally, once the cube was correctly placed on the right platform, the cube disappeared, and another trial started. Three different cube sizes were randomly presented in order to prevent subject adaptation to the cube dimensions. Each cube size was associated to a cube color. The mass of the cubes was kept constant, and the same amount of force was required to lift the cube compensating the simulated gravity force. No visual deformation of the cubes was implemented. Hence, once the virtual proxies were in contact with the cube, modulation of the grasping force could not be judged from visual feedback.

The subject’s index and thumb fingers were represented by two spheres in the virtual scenario (proxies). Coupling between the real tracked position of subject’s fingers and the position of the virtual proxies was implemented through a virtual spring. Physical simulation and virtual couplings were implemented using the NVIDIA PhysX physical engine. The frame rate was 60 fps, while physical simulation was run at 120 Hz, according to the optical tracker update frequency. With the used physical simulation parameters, the minimum indentation required to lift the cube was 1.5 mm below the contact threshold, while the virtual maximum indentation corresponded to 6 mm. The values were chosen through an exploratory experimental activity, in order to make the task challenging and enhance performance differences among the different types of feedback.

Figure 3 shows the superimposition of a grasped virtual cube to the user’s hand performing the grasping. δ₁ and δ₂, shown in the figure, represent the indentation depth of the real fingers position inside the virtual cube and are measured as the distance between the tracked finger position and the virtual proxy.

FIGURE 3

FIGURE 3. Representation of the grasped virtual cube superimposed to the user’s hand performing the grasping. δ₁ and δ₂ shown in the picture are the indentation depths of the real thimbles inside the virtual cube. They are measured as distances of the thimbles with respect to the physically constrained proxies (green and violet spheres in the picture).

An optical tracking system (Optitrack, V120 Trio) was used to track the index and thumb fingertip positions (Figure 4). Relative precision of the tracking system was assessed by fixing two markers on a rigid bar (4 cm distance, similar to the interfinger distance in the experimental condition). With the bar in a fixed position, the standard deviation of the tracked distance between the two markers was equal to 0.03 mm. We then moved the bar along a similar trajectory of the pick-and-place task proposed in the experiment. Under this condition, the standard deviation was equal to 0.11 mm with a maximum error equal to ± 0.2 mm.

FIGURE 4

FIGURE 4. Experimental setup including the virtual pick-and-place task presented on a screen, the wearable haptic devices, and an optical tracker.

The Haptic Thimble Devices

Two haptic thimble devices have been used for rendering the three different force feedback strategies described in Section 2. In order to obtain the most clean haptic feedback, an electromagnetic voice coil was used to actuate a moving plate in contact with the finger. The actuated core of the thimble was the same as the one preliminarily presented in Gabardi et al., (2016). In order to enhance transparency of the device, the coil and the contact plate composed the moving part of the actuator, while the heavier ferromagnetic core and magnet were fixed to the external structure of the thimble. Design considered 1 mm clearance between the out-of-contact position and the contact threshold with the finger and 2 mm maximum normal deformation of the fingerpad during operation in the (0–0.5) N range of force. Actuator’s geometry was designed in order to minimize variations in the output force characteristic depending on the position of the moving coil; the axial length of the coil was designed 2 mm higher than the internal polar expansion of the magnetic circuit. Two lateral linear guides, mounting low-friction polymer bushings, were designed to guide the linear movement of the coil.

The haptic thimbles were driven through a Texas Instruments DRV8835 compact H-bridge IC, capable of high pulse-width-modulation (PWM) frequencies (150 kHz frequency was used). The control algorithm of the two devices was run on a microcontroller board (PJRC Teensy 3.6) at a 1 kHz loop rate. The control board was connected to the host PC running the virtual environment via USB communication protocol.

The haptic thimbles were tested at the bench in order to measure the output force characteristic and the output bandwidth. The experimental setup for the bench test is shown in Figure 5A. The actuator was disassembled from the thimble and fixed in front of a miniaturized force sensor. The force sensor was an OptoForce OMD-10-SE-10N (three axes, range 10 N, resolution 2 mN), an optical force sensor with compact dimensions (10 mm diameter silicone rubber hemisphere). The distance between the actuator and the sensor could be adjusted in order to measure the output force characteristic at different positions of the coil. For measuring the output bandwidth of the device, the coil was actuated using a chirp voltage reference (range 10–1,000 Hz). For this specific test, reference was generated on board the controller, and the loop rate was increased to 2 kHz.

FIGURE 5

FIGURE 5. Bench test measuring the output bandwidth of the voice coil implemented in the haptic thimble (A) and the Bode plot of the measured output force (B).

The Experimental Procedure

A total of ten subjects aged between 24–38 (mean = 31; SD = 4.3) have been involved in the study and performed the experiment. An additional subject was excluded by the experiments since in the familiarization phase the subject was not able to accomplish the proposed task under any feedback condition. The experiment was conducted according to Declaration of Helsinki, and all the subjects provided written consent to participate in the study. The study was approved by the Ethical Board of the Scuola Superiore Sant’Anna of Pisa, Italy (approval number 15–2021). Each subject was seated on a comfortable chair, in front of a desktop and a computer monitor. The haptic thimbles were worn at the index and thumb fingers. Depending on the finger size, soft pads were used between the thimble and the finger dorsum to calibrate the distance between the plate and the fingerpad. The experimental setup is shown in Figure 4.

For each subject, the experiment was structured as follows: an initial phase of 5 min was presented to familiarize with the virtual task and to experience the different feedback conditions. Successively, the subject was asked to perform a session of 40 trials (pick-and-place of 40 virtual cubes) for each of the three force feedback strategies plus the only visual condition. The four different sessions were presented to the subject in a randomized order. During the visual condition, the haptic devices were worn by the subject but deactivated. After each session, the subject was asked to answer a questionnaire composed of the following questions:

• Q1: How much did you rely on the tactile feedback?

• Q2: How much did you rely on the visual feedback?

• Q3: How much was the haptic feedback realistic?

• Q4: How much were the haptic feedback and visual feedback congruent?

• Q5: How much did you manage to precisely modulate the contact force?

Concerning the only visual feedback session, only answer to Q5 has been requested. The subjects answered the questions by marking an integer value on a visual scale ranging from −3 to 3. On the same scale, it has been finally asked to the subjects to answer the following questions concerning the comparison of the four kinds of force feedback tried:

• G1: For each kind of feedback, rate your overall preference.

• G2: For each kind of feedback, rate your confidence in accomplishing the task.

• G3: For each kind of feedback, rate the perceived difficulty in accomplishing the task.

Data Analysis

A total of four performance measures were extracted from the data acquired during the experiment. In particular, for each trial of each subject, only the portion of data in which the subject held the object was used for measuring the indentation of the two fingers within the virtual object, proportional to the virtual grasping force. The average, the standard deviation, and the maximum displacement have been computed in order to give a measurement of the stability of the grasp. Moreover, the correct rate was extracted for each subject by dividing the number of successful trial by the number of all trials. A one-way ANOVA test was conducted separately for each of the four performance metrics, after determining the normality of the data distribution using the Lilliefors test. Post hoc comparisons were corrected using the Bonferroni method. Regarding the questionnaire, the results were analyzed using the nonparametric Friedman test separately for each question. When a significant effect was found, the multiple comparisons were assessed through the Wilcoxon signed-rank test corrected with the Bonferroni method.

Results

The graph in Figure 6A summarizes the simulated results of the current-to-force characteristic for different positions and different values of the current intensity. The slope of each line represents the force coefficient of the actuator at different positions of the moving coil. The results in Figure 6B show that within the designed range of motion (3 mm), the actuator output force has a variability below 0.01%. The results of the characterization at the bench of the haptic thimble are shown in Figure 5. The measured Bode plot of the output force shows an output bandwidth from 0 to 275 Hz.

FIGURE 6

FIGURE 6. Simulated output force characteristic of the voice coil at different positions of the coil (A). Comparison of simulated (red) and measured (blue) output force at different positions of the coil (B).

We also conducted an estimation of the energy dissipated in the actuators’ coils under the D and P conditions. Since the actuators were driven in feed-forward, the estimation had been conducted using the voltage reference signal, thus neglecting BEMF and inductance effects. The dissipated energy ratio between the D and P conditions, computed across all trials and all subjects, was below 0.005. The apparently very low value can be explained by the fact that p power dissipated under the continuous feedback condition is highly affected by duration of the grasping action. On the other hand, D feedback is active only during transients, and even there, each pulse has very limited energy consumption (proportional to the 1 ms duration of each pulse).

Figure 7 reports the correct ratio of the pick-and-place task and three metrics related to the grasping indentation measured during the experiment under the four feedback conditions. The three metrics correspond to the mean value of the indentation depth (higher value corresponds to higher virtual force exerted on the object), stability of the indentation depth (computed as the standard deviation of the indentation depth during each repetition), and peak value of the indentation depth, measured as the maximum value for each repetition. A one-way ANOVA with post hoc analysis was conducted over the correct ratio and the aforementioned metrics. The results related to the correct ratio show a significant main effect of the provided feedback (F (3) = 17.7, p < 0.001). Bonferroni post hoc tests showed that the participant’s correct rate was significantly lower under the V condition (mean = 0.69; SD = 0.17) than that under all the other feedback conditions (P mean = 0.76; SD = 0.15; p < 0.001), (D mean = 0.58; SD = 0.23; p < 0.05), and (PD mean = 0.70; SD = 0.20; p < 0.001). The P feedback correct ratio was significantly higher than the D feedback (p < 0.01). The correct ratio between the PD and D feedback was not significantly different. A similar result was obtained for the grasping indentation variability metric, with all the feedback conditions reporting a significantly lower variability than the V condition (V mean = 0.96, SD = 0.13, P mean = 0.78, SD = 0.08, p < 0.01; D mean = 0.79, SD = 0.16, p < 0.01; PD mean = 0.77, SD = 0.12, and p < 0.01). Differences were not significant between different haptic feedback conditions. Regarding the grasping indentation mean, differences were found significant only between the V and PD condition (V mean = 2.55, SD = 0.44; PD mean = 2.22, SD = 0.29, p < 0.01) and between the PD and P condition (P mean = 2.51, SD = 0.48, p < 0.05). The peak grasping indentation metric showed a significant difference only between the V and PD condition (V mean = 4.08, SD = 0.65; PD mean = 3.38, SD = 0.48, p < 0.01).

FIGURE 7

FIGURE 7. Correct ratio of the pick-and-place task, mean, stability (standard deviation), and peak value of the grasping indentation.

Figure 8 (first row) shows the averaged answers (median with 25th and 75th percentiles) of the questions related to the haptic feedback. In Figure 8 (second row) are reported the answers to the questions regarding the overall experience. As can be noted in Figure 8, questions ranging from Q1 to Q4 did not include the V condition. The Friedman test provided evidence of a difference (χ² (2, 18) = 8.97, p = 0.011) between the answers to the question Q3, “the haptic feedback was realistic.” Post hoc analysis with Wilcoxon signed-rank tests was conducted with a Bonferroni correction applied, resulting in a significance level set at p $<0.017$. There was a significant difference between the P condition [median = 7.0 (7.0 7.0)] and the D condition [median = 5 (4.0 6.0)]. A significant difference was found also in answers to question Q5, “did you manage to precisely modulate the contact force?” (χ² (3, 27) = 21.79, p$<0.001$). The post hoc test, corrected with Bonferroni and resulting in a significant level set at p < 0.008 3, evidenced the fact that scores given to the V feedback [median = 2 (2.0 4.0)] were significantly lower than those given to the other feedback conditions for both the comparison between the P and PD feedback (p < 0.01) and between P and D feedback (p < 0.01). The similar results were found for the G1 and G2 showing a significant main effect (p$<0.001$) with a lower score given to the V condition with respect to all the other feedback conditions. Regarding the perceived difficulty of the task, question G3, a significant main effect was found as well (χ² (3, 27) = 22.62, p$<0.001$). The post hoc comparisons, corrected with the Bonferroni method, evidenced a significant difference only between the V condition and the P condition (p < 0.01).

FIGURE 8

FIGURE 8. Answers to the questionnaire. First row reports the answer to the questions related to the three implemented haptic feedback. Second row reports the answers to the questions related to the perceived performance over the task under the four different experimental conditions.

Discussion

The first experimental activity aimed at characterizing the haptic actuators used in the subject study, ensuring that the experimented feedback could be properly rendered to the subjects. An electromagnetic voice coil linear actuator was chosen as the most suitable actuator typology for a comparison experiment of different feedback signals; although heavier than alternative actuator designs, it presents, in general, a transparent response in a wide range of frequencies, including static output force, due to the absence of reduction or transformation mechanisms. The measured frequency response showed a band-pass frequency from 0 to 275 Hz, covering the range of frequencies required for cutaneous haptic feedback. Regarding the current-to-force characteristic, we designed the actuator geometry in order to keep such relationship constant in the used workspace. This was obtained by designing the length of the coil longer than the length polar expansion of the magnetic core. Measured output force at different positions of the moving plate well matched the simulated behavior of the actuator. Considering 3 mm stroke, variability of the measured output force is below 0.1%.

Regarding results of the virtual pick-and -place task, the correct rate represents the overall effectiveness of the feedback in accomplishing the proposed task. All the haptic feedback signals performed significantly better than the visual feedback alone, showing that both the P and the proposed D feedback were coherent and informative of the performed task. The grasping indentation variability reported coherent results, showing that during each manipulation task finger position and indentation with the virtual object was more stable for all the haptic feedback condition than the visual condition. Again for the correct ratio, the P feedback reported the higher performance, with a significant difference also than D feedback. It showed that the continuous feedback performed better than the proposed discrete one in terms of overall results. The possible reasons are in the continuous modulation of the feedback, achieving a higher rendering resolution of the grasping force modulation with respect to the discrete transients of the D feedback. It also allows to discriminate the sign of variation of the grasping force, whereas discrete transients of the D feedback do not carry information about increasing or decreasing variation of the grasping force. Furthermore, the P feedback resembles the natural perception of the grasping force modulation, thus possibly becoming immediate to be effectively used by users, without adaptation or training.

On the other hand, the D feedback resulted significantly better than the V condition, suggesting that in applications where high wearability and lightweight of the haptic devices are needed, it might be profitably used. As explained in Section 1, by excluding the constant and slow force components by the rendering, power required by actuators is significantly lower. Furthermore, the actuator design and typology can be conceived differently, if only high frequencies have to be rendered, i.e., flexure hinges or membranes can be used to hold the moving part of voice coil actuators, in place of guides or joints, and different actuator typologies (i.e., piezo actuators) can be used, since the required stroke is considerably lower.

It is interesting to note that the superposition of the two feedback strategies under the condition PD obtained an average score for the correct ratio similar to the separated P and D feedback (with no significant differences). It suggests that the superposition of the two feedback strategies does not negatively interfere with effectiveness in the manipulation task; further investigation to improve quality of the D feedback might involve the superposition of a continuous feedback component, with reduced intensity, to the D feedback. The trade-off between effectiveness of the feedback and the required total energy of the signal should be explored. The results of the questionnaire answers of the haptic questions group showed a better performance of the P feedback with respect to the D feedback (significant for Q3). Findings are coherent with the fact that the continuous feedback is the most similar with the natural haptic perception in manipulation tasks. The results of the answers to the task-related questions group showed a better performance in terms of preferences, perceived control, and perceived difficulty of the task for all the proposed feedback with respect to the visual condition. With similar considerations to the aforementioned results, we can conclude that the proportional feedback (P) provided the overall best performance in terms of preferences and of perceived and measured effectiveness. Still, the proposed discrete feedback (D) performed significantly better than the visual condition, suggesting its possible use in applications where other advantages, in terms of reduced actuators dimensions and typology, are more relevant.

The experiment involved a discrete feedback implemented in its simplest form (a train of pulses) to obtain a first robust experimental comparison with conventional continuous feedback in a grasping task. Yet, more advanced versions can be considered and experimented. In particular, we are interested in experimenting a version of the feedback capable of conveying also the sign of the force variation. This can be envisaged in different ways, i.e., by modulating amplitude or frequency content of each discrete cue, in order to discriminate the sign of the indentation increment.

Finally, the estimation of the energy consumption ratio between the D and P feedback conditions is meaningful of the potential of the discrete feedback in terms of the smaller device design. We clarify that the noticeably high difference in energy consumption does not translate in a proportional reduction of the actuator size, and it would be difficult to generalize a scaling factor without a finalized device design. Still, heat dissipation is a critical element concurring in determining the final size of an electromagnetic actuator.

Conclusion

In this article, we investigated a strategy for rendering haptic feedback, informative of grasping force modulation, by means of discrete, high-frequency cues. Perception of the increasing grasping force was provided by force pulses triggered at discrete indentation thresholds. The envisaged advantage of the proposed discrete feedback was to considerably reduce the continuous output force demanded to the actuator, which in turn considerably reduces the overall dimensions of the actuator. This becomes critical in fingertip haptic devices, in which mass and size have to be limited to improve wearability and preserve dexterity, in particular for certain training and rehabilitation scenarios (i.e., Bortone et al. (2020)), in which users are children with reduced hand dimensions if compared to adult users). We experimentally evaluated effectiveness of the discrete feedback by measuring objective performance metrics in a virtual pick-and-place task of fragile objects. We compared the discrete feedback (D) with the conventional continuous proportional feedback (P), with a superposition of the two feedback strategies (PD) and with the visual only condition. The experimental setup and the implemented wearable haptic devices based on voice coil actuators were optimized for improving the quality of the feedback presented to participants.

The obtained results showed an overall better performance of the conventional continuous feedback, both in terms of users’ perception and objective measurements. The correct ratio of the given pick-and-place task was significantly better for the P feedback, and answers to the feedback-related questions evidenced the P feedback was perceived significantly more natural than the D feedback. While the P feedback resembles a natural perception of the grasping force modulation, the D feedback tries to carry similar information through a different modulated signal. To this regard, an aspect to consider in future investigations is how learning might improve effectiveness of the D feedback in the objective metrics of the manipulation tasks. On the other hand, all the evaluated feedback performed significantly better than the visual condition only in terms of correct ratio, stability of the grasping pose, perceived difficulty of the task, and perceived precision in grasping modulation. This envisaged the use of the proposed D feedback in applications involving a convenient trade-off between feedback performance and physical requirements of the haptic devices, in terms of limited size and mass. The advantages of the D feedback extend also to different actuator designs that can be considered, requiring rendering of high-frequency signals only with limited stroke of the moving parts and no static components. Future investigations can be focused also on more advanced variations of the proposed discrete feedback; different discrete cues than pulses can be studied to convey additional information, such as the direction of the grasping force modulation.

Data Availability Statement

The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Board of Ethics of the Scuola Superiore Sant'Anna of Pisa, Italy https://www.santannapisa.it/en/university/board-ethics. The participants provided their written informed consent to participate in this study.

Author Contributions

DL and MG contributed to the development of both the haptic thimble devices and the experimental hardware/software setup. All the authors equally contributed to the design of the experiments and the data collection. Data had been analyzed by MB. The work has been revised by AF and funded by AF and DL.

Funding

This work was supported by the project “TELOS—Tailored neurorehabilitation thErapy via multi-domain data anaLytics and adapative seriOus games for children with cerebral palSy”, which is funded under the call “Bando Ricerca Salute 2018” of Tuscany Region, Italy (CUP J52F20001040002).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors, and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Basdogan, C., and Srinivasan, M. A. (2002). “Haptic Rendering in Virtual Environments,” in Handbook of virtual environments 1.

Google Scholar

Bensmaıa, S. J., and Hollins, M. (2000). Complex Tactile Waveform Discrimination. The J. Acoust. Soc. Am. 108, 1236–1245.

Google Scholar

Bergamasco, M., Avizzano, C. A., Frisoli, A., Ruffaldi, E., and Marcheschi, S. (2006). Design and Validation of a Complete Haptic System for Manipulative Tasks. Adv. Robotics 20, 367–389. doi:10.1163/156855306776014367

CrossRef Full Text | Google Scholar

Bortone, I., Leonardis, D., Solazzi, M., Procopio, C., Crecchi, A., Bonfiglio, L., et al. (2017). “Integration of Serious Games and Wearable Haptic Interfaces for Neuro Rehabilitation of Children with Movement Disorders: A Feasibility Study,” in 2017 International Conference on Rehabilitation Robotics (ICORR), London, United Kingdom, July 17–20, 2017 (IEEE), 1094–1099. doi:10.1109/ICORR.2017.8009395

PubMed Abstract | CrossRef Full Text | Google Scholar

Bortone, I., Barsotti, M., Leonardis, D., Crecchi, A., Tozzini, A., Bonfiglio, L., et al. (2020). Immersive Virtual Environments and Wearable Haptic Devices in Rehabilitation of Children with Neuromotor Impairments: a Single-Blind Randomized Controlled Crossover Pilot Study. J. Neuroeng Rehabil. 17 (1), 1–14. doi:10.1186/s12984-020-00771-6

CrossRef Full Text | Google Scholar

Caldwell, D. G., Tsagarakis, N., and Wardle, A. (1997). “Mechano Thermo and Proprioceptor Feedback for Integrated Haptic Feedback,” in International Conference on Robotics and Automation, Albuquerque, ID, April 25–27, 1997, 2491–2496. Robotics and Automation.3

Google Scholar

Cappello, L., Alghilan, W., Gabardi, M., Leonardis, D., Barsotti, M., Frisoli, A., et al. (2020). Continuous Supplementary Tactile Feedback Can Be Applied (And Then Removed) to Enhance Precision Manipulation. J. Neuroeng Rehabil. 17 (1), 1–13. doi:10.1186/s12984-020-00736-9

CrossRef Full Text | Google Scholar

Chinello, F., Malvezzi, M., Pacchierotti, C., and Prattichizzo, D. (2015). “Design and Development of a 3rrs Wearable Fingertip Cutaneous Device,” in International Conference on Advanced Intelligent Mechatronics (AIM), Busan, Korea, July 7–11, 2015, 293–298. doi:10.1109/aim.2015.7222547

CrossRef Full Text | Google Scholar

Cipriani, C., Segil, J. L., Clemente, F., ff. Weir, R. F., and Edin, B. (2014). Humans Can Integrate Feedback of Discrete Events in Their Sensorimotor Control of a Robotic Hand. Exp. Brain Res. 232, 3421–3429. doi:10.1007/s00221-014-4024-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Clemente, F., D'Alonzo, M., Controzzi, M., Edin, B. B., and Cipriani, C. (2016). Non-invasive, Temporally Discrete Feedback of Object Contact and Release Improves Grasp Control of Closed-Loop Myoelectric Transradial Prostheses. IEEE Trans. Neural Syst. Rehabil. Eng. 24, 1314–1322. doi:10.1109/tnsre.2015.2500586

PubMed Abstract | CrossRef Full Text | Google Scholar

Cole, K. J., and Abbs, J. H. (1988). Grip Force Adjustments Evoked by Load Force Perturbations of a Grasped Object. J. Neurophysiol. 60, 1513–1522. doi:10.1152/jn.1988.60.4.1513

CrossRef Full Text | Google Scholar

Fani, S., Ciotti, S., Battaglia, E., Moscatelli, A., and Bianchi, M. (2017). W-fyd: A Wearable Fabric-Based Display for Haptic Multi-Cue Delivery and Tactile Augmented Reality. IEEE Trans. Haptics 11, 304–316. doi:10.1109/TOH.2017.2708717

PubMed Abstract | CrossRef Full Text | Google Scholar

Gabardi, M., Solazzi, M., Leonardis, D., and Frisoli, A. (2016). “A New Wearable Fingertip Haptic Interface for the Rendering of Virtual Shapes and Surface Features,” in Haptics Sympoisum 2016, Philadelphia, PA, April 8–11, 2016 (IEEE), 140–146. doi:10.1109/haptics.2016.7463168

CrossRef Full Text | Google Scholar

Gabardi, M., Chiaradia, D., Leonardis, D., Solazzi, M., and Frisoli, A. (2018). “A High Performance thermal Control for Simulation of Different Materials in a Fingertip Haptic Device,” in Eurohaptics 2018, Pisa, Italy, June 13–16, 2018 (Springer), 313–325. doi:10.1007/978-3-319-93399-3_28

CrossRef Full Text | Google Scholar

Gallo, S., Rognini, G., Santos-Carreras, L., Vouga, T., Blanke, O., and Bleuler, H. (2015). Encoded and Crossmodal thermal Stimulation through a Fingertip-Sized Haptic Display. Front. Robot. AI 2, 25. doi:10.3389/frobt.2015.00025

CrossRef Full Text | Google Scholar

Gutiérrez, Á., Farella, N., Gil-Agudo, Á., and de los Reyes Guzmán, A. (2021). Virtual Reality Environment with Haptic Feedback Thimble for post Spinal Cord Injury Upper-Limb Rehabilitation. Appl. Sci. 11, 2476. doi:10.3390/app11062476

CrossRef Full Text | Google Scholar

Kildal, J. (2010). “3d-press: Haptic Illusion of Compliance when Pressing on a Rigid Surface,” in International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction, Beijing, China, November 8–10, 2010 (New York, NY: ACM Publications), 21.

Google Scholar

Klamt, T., Schwarz, M., Lenz, C., Baccelliere, L., Buongiorno, D., Cichon, T., et al. (2020). Remote mobile Manipulation with the Centauro Robot: Full‐body Telepresence and Autonomous Operator Assistance. J. Field Robotics 37, 889–919. doi:10.1002/rob.21895

CrossRef Full Text | Google Scholar

Kuchenbecker, K. J., Fiene, J., and Niemeyer, G. (2006). Improving Contact Realism through Event-Based Haptic Feedback. IEEE Trans. Vis. Comput. Graph 12, 219–230. doi:10.1109/TVCG.2006.32

PubMed Abstract | CrossRef Full Text | Google Scholar

Kuchenbecker, K. J., Gewirtz, J., McMahan, W., Standish, D., Martin, P., Bohren, J., et al. (2010). “Verrotouch: High-Frequency Acceleration Feedback for Telerobotic Surgery,” in IEEE Transactions on Visualization and Computer Graphics (Springer), 189–196. doi:10.1007/978-3-642-14064-8_28

CrossRef Full Text | Google Scholar

Kurihara, Y., Hachisu, T., Sato, M., Fukushima, S., and Kajimoto, H. (2013). “Periodic Tactile Feedback for Accelerator Pedal Control,” in Eurohaptics 2010, Amsterdam, Netherlands, July 8–10, 2010 (IEEE), 187–192. doi:10.1109/whc.2013.6548406

CrossRef Full Text | Google Scholar

Leonardis, D., Solazzi, M., Bortone, I., and Frisoli, A. (2015). “A Wearable Fingertip Haptic Device with 3 Dof Asymmetric 3-rsr Kinematics,” in IEEE World Haptics Conference, Chicago, IL, June 22–26, 2015 (IEEE), 388–393. doi:10.1109/whc.2015.7177743

CrossRef Full Text | Google Scholar

Leonardis, D., Solazzi, M., Bortone, I., and Frisoli, A. (2017). A 3-rsr Haptic Wearable Device for Rendering Fingertip Contact Forces. IEEE Trans. Haptics 10, 305–316. doi:10.1109/toh.2016.2640291

PubMed Abstract | CrossRef Full Text | Google Scholar

Leonardis, D., Gabardi, M., Solazzi, M., and Frisoli, A. (2020). “A Parallel Elastic Haptic Thimble for Wide Bandwidth Cutaneous Feedback,” in Eurohaptics 2020, Leiden, Netherlands, September 6–9, 2020 (Springer), 389–397. doi:10.1007/978-3-030-58147-3_43

CrossRef Full Text | Google Scholar

Leonardis, D., Tiseni, L., Chiaradia, D., and Frisoli, A. (2021). A Twisted String, Flexure Hinges Approach for Design of a Wearable Haptic Thimble. Actuators 10, 211. doi:10.3390/act10090211

CrossRef Full Text | Google Scholar

Maisto, M., Pacchierotti, C., Chinello, F., Salvietti, G., De Luca, A., and Prattichizzo, D. (2017). Evaluation of Wearable Haptic Systems for the Fingers in Augmented Reality Applications. IEEE Trans. Haptics 10, 511–522. doi:10.1109/toh.2017.2691328

PubMed Abstract | CrossRef Full Text | Google Scholar

Niemeyer, G., and Mitra, P. (2005). “3 Dynamic Proxies and Haptic Constraints,” in Multi-point Interaction with Real and Virtual Objects (Springer), 41–53.

Google Scholar

Pacchierotti, C., Prattichizzo, D., and Kuchenbecker, K. J. (2016). Cutaneous Feedback of Fingertip Deformation and Vibration for Palpation in Robotic Surgery. IEEE Trans. Biomed. Eng. 63, 278–287. doi:10.1109/TBME.2015.2455932

PubMed Abstract | CrossRef Full Text | Google Scholar

Pacchierotti, C., Prattichizzo, D., and Kuchenbecker, K. J. (2016). Cutaneous Feedback of Fingertip Deformation and Vibration for Palpation in Robotic Surgery. IEEE Trans. Biomed. Eng. 63, 278–287. doi:10.1109/tbme.2015.2455932

PubMed Abstract | CrossRef Full Text | Google Scholar

Pacchierotti, C., Sinclair, S., Solazzi, M., Frisoli, A., Hayward, V., and Prattichizzo, D. (2017). Wearable Haptic Systems for the Fingertip and the Hand: Taxonomy, Review, and Perspectives. IEEE Trans. Haptics 10, 580–600. doi:10.1109/toh.2017.2689006

PubMed Abstract | CrossRef Full Text | Google Scholar

Prattichizzo, D., Pacchierotti, C., and Rosati, G. (2012). Cutaneous Force Feedback as a Sensory Subtraction Technique in Haptics. IEEE Trans. Haptics 5, 289–300. doi:10.1109/toh.2012.15

PubMed Abstract | CrossRef Full Text | Google Scholar

Robles-De-La-Torre, G., and Hayward, V. (2001). Force Can Overcome Object Geometry in the Perception of Shape through Active Touch. Nature 412, 445–448. doi:10.1038/35086588

PubMed Abstract | CrossRef Full Text | Google Scholar

Schorr, S. B., Quek, Z. F., Romano, R. Y., Nisky, I., Provancher, W. R., and Okamura, A. M. (2013). “Sensory Substitution via Cutaneous Skin Stretch Feedback,” in Robotics and Automation (ICRA), 2013 IEEE International Conference on (IEEE), 2341–2346. doi:10.1109/icra.2013.6630894

CrossRef Full Text | Google Scholar

Solazzi, M., Frisoli, A., and Bergamasco, M. (2010). “Design of a Novel finger Haptic Interface for Contact and Orientation Display,” in IEEE Haptics Symposium, Waltham, MA, March 25–26, 2010 (IEEE), 129–132. doi:10.1109/haptic.2010.5444667

CrossRef Full Text | Google Scholar

Stepp, C. E., and Matsuoka, Y. (2011). Vibrotactile Sensory Substitution for Object Manipulation: Amplitude versus Pulse Train Frequency Modulation. IEEE Trans. Neural Syst. Rehabil. Eng. 20, 31–37. doi:10.1109/TNSRE.2011.2170856

PubMed Abstract | CrossRef Full Text | Google Scholar

Strohmeier, P., Boring, S., and Hornbæk, K. (2018). “From Pulse Trains to” Coloring with Vibrations” Motion Mappings for Mid-air Haptic Textures,” in CHI Conference on Human Factors in Computing Systems, Montreal, Canada, April 21–26, 2018, 1–13.

Google Scholar

Visell, Y., Duraikkannan, K. A., and Hayward, V. (2014). “A Device and Method for Multimodal Haptic Rendering of Volumetric Stiffness,” in Eurohaptics 2014, Versailles, France, June 24–27, 2014 (Springer), 478–486. doi:10.1007/978-3-662-44193-0_60

CrossRef Full Text | Google Scholar

Wang, D., Ohnishi, K., and Xu, W. (2019). Multimodal Haptic Display for Virtual Reality: A Survey. IEEE Trans. Ind. Electron. 67, 610–623. doi:10.1109/TIE.2019.2920602

CrossRef Full Text | Google Scholar

Westling, G., and Johansson, R. S. (1987). Responses in Glabrous Skin Mechanoreceptors during Precision Grip in Humans. Exp. Brain Res. 66, 128–140. doi:10.1007/BF00236209

PubMed Abstract | CrossRef Full Text | Google Scholar

Wiertlewski, M., Lozada, J., and Hayward, V. (2011). The Spatial Spectrum of Tangential Skin Displacement Can Encode Tactual Texture. IEEE Trans. Robot. 27, 461–472. doi:10.1109/tro.2011.2132830

CrossRef Full Text | Google Scholar

Yau, J. M., Olenczak, J. B., Dammann, J. F., and Bensmaia, S. J. (2009). Temporal Frequency Channels Are Linked across Audition and Touch. Curr. Biol. 19, 561–566. doi:10.1016/j.cub.2009.02.013

PubMed Abstract | CrossRef Full Text | Google Scholar