Reality bites: highlighting the potential discrepancies between multisensory taste perception in extended and physical reality

Introduction: When we taste, we take in a variety of sensory information that can be completely independent from the properties of the food itself: the ambient lighting and environmental sounds can all influence our taste perception and affective responses. However, current multisensory research is mixed as to whether these In Real Life (IRL) findings also apply to Extended Reality (XR) environments. A potential reason for this may be the limited realism of some XR scenarios, which this study aimed to overcome through an immersive Virtual Reality experience (VR, chosen for its greater realism relative to other XR applications) whilst also expanding the scope to flavour perception.

Methods: A total of 34 participants rated food samples under neutral, red, and green ambient lighting in VR. Participants ate either lime flavoured, strawberry flavoured, or “Neutral” (no added flavour) samples.

Results: While participants were equally immersed in all three environments, they rated the red and green lighting environments as substantially less natural than the neutral lighting environment. Interestingly, while participants associated sweetness and sourness with red lighting and green lighting respectively, this minimally extended to their behavioural ratings, when sampling the stimuli in VR. Samples eaten under red lighting were rated as significantly sweeter than those eaten under neutral lighting. However, neither red nor green lighting affected sample sourness nor the intensity of strawberry or lime flavour.

Discussion: This study found limited evidence of multisensory integration in XR, wherein taste expectations do not extend to taste experiences. We discuss these results in light of prior works on crossmodal associations and raise a reflection on why multisensory integration of taste may not apply to XR in our study. While there is much excitement about the opportunities XR can offer, we argue that we are only at the beginning of understanding the experiences on the reality-virtuality continuum and that we need to establish a richer understanding of participants' experiences, expectations, and taste/flavour perceptions between IRL and XR.

1 Introduction

Our gustatory experiences, while often assumed to be entirely a result of the food that comes into contact with our tongues, are far more complex, comprehensive, and interesting. A well-known example is how critical smell is for our experience of flavour, which is readily apparent when we are sick or if we were to simply hold our nose when eating. However, our eating experiences are not limited to only our chemical senses. It is shaped by our taste expectations and multisensory integration (Velasco and Obrist, 2021), even when this information is completely irrelevant to the food itself. Evidence that visual cues can influence taste perception can be divided into intrinsic factors that involve attributes that alter the food appearance itself and extrinsic factors which involve modifying cues in the environment that are related to the food (Piqueras-Fiszman and Spence, 2015; Carvalho and Spence, 2019). A clear example of a food intrinsic factor is the colour of the food (Spence, 2015). colour is a key indicator of food quality and freshness and creates a learned expectation of taste that can prime a flavour experience. For example, the intensity of red colour in food is associated with perceived spiciness independent of actual food spice level (Levitan and Shermer, 2014). The texture of food is another intrinsic factor that can affect our expectations. Creamy textures, for example, might be associated with a rich, luxurious flavour (Tarrega et al., 2016), firm fruits may hint at a less intense sweetness, and crunchy textures and sounds can signify freshness and can make the food seem more pleasant (Vickers, 1983).

While some of these expectations may be learned, there is consistent evidence that our expectations are also influenced by crossmodal correspondences: the associations that people make between different sensory modalities. A classic example is the kiki-bouba effect, in which the word bouba is described as sounding “sweet” and “round” whereas kiki is associated with “sour” and “sharp” (Köhler, 1929; Salgado-Montejo et al., 2015; Velasco et al., 2015). This also extends to the handling of bouba-shaped and kiki-shaped stimuli (Graven and Desebrock, 2018), rough vs. smooth textures (Etzi et al., 2016; Slocombe et al., 2016; Riofrio-Grijalva et al., 2020), the presentation style of food (Fairhurst et al., 2015) and the actual mouthfeel of the food itself (Cornelio et al., 2022).

What is especially interesting is how food extrinsic cues (i.e., those that do not determine the physical properties of food) can influence perception and affective experiences. For example, the physical properties of cutlery and glasses such as their weight can influence the perceived properties of food and beverages (Piqueras-Fiszman and Spence, 2011; Harrar and Spence, 2013), wine can taste more refined with a corresponding background musical style (North, 2012), pink drinking cups increase the expected and reported sweetness of coffee (Carvalho and Spence, 2019), and even the colouring of cigarette packaging can affect the perceived harm of the cigarettes themselves (Brown et al., 2021). There have been numerous studies that have also explored how the design of eating environments influences our experiences. Previous studies have shown relatively consistent evidence that red-coloured environments are associated with sweetness. For example, Velasco et al. (2013) reported that whisky tasted under red lighting was reported to be sweeter, but fresher under green lighting. Similarly, Spence et al. (2014) reported that wine was reported as fresher under green lighting and while listening to “sour” music, with enjoyment highest under red lighting and “sweet” music. More recently, Istiani et al. (2022) reported that red ambient lighting increases the sweetness and aroma ratings of juice relative to green lighting.

Considering the above, Extended Reality (XR) technology offers a unique opportunity to manipulate the dining environment in ways that are not possible in the physical world (Liu et al., 2019). For instance, the colour of the environment can be changed instantly and without any physical constraints, allowing researchers to investigate its impact on taste and flavour perception in a controlled and systematic way. This is significant because it opens up new possibilities for enhancing the dining experience, from creating immersive themed dining environments to personalising the environment based on individual preferences. This domain of XR Human-Food Interaction (HFI) is relatively novel and the design space has only begun to be formalised (Velasco and Obrist, 2021), yet to take advantage of the rich knowledge of IRL crossmodal correspondences. Currently, XR studies have superimposed a different (incongruent) flavour onto food (Spence, 2023) or enhanced taste qualities through XR (Duffy et al., 2009; Kurihara, 2015), known as “sonic seasoning” (Spence and Youssef, 2019). These environments have also been found to influence decision-making and appraisals. For example, through an increased willingness to pay for and higher enjoyment of wine in a virtual winery compared to a virtual bar (Picket and Dando, 2019) and choosing drinks congruent with current digital scenarios (e.g., coffee, tea, or hot chocolate in warm environments, Spence, 2020). A critical advantage of these techniques is that the physical attributes of the food itself can be maintained (Ueda et al., 2020). This is especially useful in cases were changing the colour or structure of food may change its taste as an unwanted consequence (e.g., bitter food colourings). Alternatively, XR can take simple physical food samples and present an immersive and pleasant multisensory experience to improve taste and flavour perception. For the food and beverage industry, understanding how the colour of the environment influences taste and flavour perception could lead to innovative dining experiences that not only satisfy the palate but also engage the other senses. For health and nutrition, this knowledge could be used to encourage healthier eating habits. For example, if certain colours are found to enhance the perception of sweetness, they could be used in environments where low-sugar foods are served to make them more appealing. Furthermore, understanding these crossmodal correspondences could also provide valuable insights into consumer behaviour, informing the design of product packaging, marketing campaigns, and retail spaces.

However, evidence for the effects of crossmodal correspondences on taste and flavour within the XR literature is inconsistent. In terms of food intrinsic factors, the “MetaCookie+” study by Narumi et al. (2011) revealed that overlaying a real cookie with a visual representation flavour (e.g., chocolate or almond) using an Augmented Reality headset (AR) increased the identification of the respective flavour, despite the physical cookie remaining constant. More recent studies have replicated such taste/flavour modulations. Nishizawa et al. (2016) reported that digitally changing the colour of chips/crisps from beige to green using projective AR changed the flavour to seaweed or green tea, while the intensity of taste qualities, particularly sweetness, was increased by increasing the chroma (intensity) of a yellow tinted cake. Similarly, Ammann et al. (2020) reported that decreasing the chroma of a yellow lemon cake towards brown decreased the correct identification of the lemon flavour—using VR and a motion tracker to locate food items. Modulating food luminosity has also been found to affect expectations of food moistness, wateriness, flavour intensity, and deliciousness (Ueda et al., 2020), as well as affecting the taste experiences of moistness and deliciousness (but not sweetness). One recent study by Huang et al. (2019) asked participants to select a colour that they associated with the taste of red tea and green tea in Virtual Reality (VR), replicating that reddish and greenish colours were associated with the respective tea. However, not all food intrinsic modulations have produced effect. For example, Weidner et al. (2023) compared crossmodal associations by pairing olfactory and visual cues while eating tasteless mash in VR. For example, the scent of tomato paired with a tomato visual. Surprisingly, participants could not correctly identify whether visual food cues were the same food item as the olfactory cues they smelled while eating a tasteless mash. Since participants rated the olfactory and visual cues as less intense in VR relative to outside VR during a pre-study, VR may have inhibited multimodal integration in this case by dampening the sensory experience.

In terms of food extrinsic factors, such as the colour of virtual environments, findings appear less consistent and scarce. In recent studies, tea has been rated as sweeter under pink lighting (Wu et al., 2022), blue environments have been associated with reduced emotional control (Cornelio et al., 2022), sweet drinks have been rated as sweeter in “sweet congruent” environments (i.e., smooth environmental design and red colour) and bitter with a rough and grainy texture in dark environments (Chen et al., 2020). However, not all past research replicates these findings and those expected from IRL research. For example, while Nygård and Lie (2020) found that white environments were linked to more flavourful coffee relative to red environments, there were no associations to coffee sweetness or acidity as expected. Similarly, the studies by Wu et al. (2022) found no associations between any other tested colour and sweetness, sourness, bitterness, or saltiness and Cornelio et al. (2022) found no association to sample taste. Interestingly, Liu et al. (2019) found that participants spent more time with their coffee in an immersive VR environment with multisensory contextual cues congruent with a coffee house (e.g., cinnamon, coffee house setting, relaxed music) compared to incongruent environments (e.g., detergent, laboratory setting, construction noise), only a small significant reduction in coffee liking occurred in the environment which combined all incongruent cues. This may suggest that multiple levels of crossmodal associations may be necessary to affect experiences in VR. Overall, current studies in XR are inconsistent in whether digital manipulation of crossmodal correspondences affects our eating experiences. A deeper understanding of how features of the environment within VR can be used to enhance consumer experience is required (Pereira-Castro et al., 2022; Spence, 2023).

In summary, while crossmodal correspondences are consistently seen in taste/flavour expectations and taste experiences of IRL studies, there is less consistent evidence in XR studies. The current study was designed to further investigate crossmodal correspondences in XR, informed by insights from our previous work. In Cornelio et al. (2022) we cautioned the implications of the results due to the limited effect sizes, which were both lower than expected and may have limited practical impact on digital food experiences. We suggested a potential reason may have been the artificialness of the red and blue lighting environments relative to the neutral lighting environment. Accordingly, a key aim of the current study was to improve the realism of the ambient lighting which may improve the translation of taste expectations to taste experiences. We also sought to expand the study to assess different sample flavours, both to create a more pleasant control sample and to understand more complex eating experiences.

Moreover, participants in our 2022 experiment did not all agree with the Red-Sweet and Blue-Less sweet association. As a result, for the current study, our approach was to first conduct an online pre-study (see below) to select colours and flavours with clear associations to sweet and sour taste. Next, we used these selected variables in a main user study wherein participants ate flavoured food samples under different ambient lighting conditions, to assess whether these expectations translate to taste experiences.

2 Pre-study

Our online pre-study aimed to replicate crossmodal correspondences of taste expectations. Specifically, we aimed to investigate the crossmodal correspondences of ambient lighting colour and both taste and flavour that would inform the design of our main study. To do so, a preliminary study was first conducted with three objectives: (1) to identify two colours associated exclusively with sweetness or sourness, (2) to select two flavours with crossmodal correspondences to these chosen colours, and (3) to identify a sample shape that did not have a disproportionate association with either sweetness or sourness. This was done to control for this potential confound in the study.

2.1 Study design

Online participants were asked to rate colours, flavours, and shapes on the five basic tastes (sweet, sour, bitter, umami, and salty). For colour, Red, Orange, Yellow, Green, Blue and Purple were rated (e.g., “To me, red is sweet” rated on a scale from 0% [not at all] to 100% [completely]). For flavour, we chose to keep items within the same category of food for consistency. Fruit flavours were chosen due to their links to sweetness and sourness, varied colours, and availability of flavouring to purchase. Fourteen different fruits were assessed (Banana, Cherry, Fig, Grapefruit, Kiwi, Lemon, Lime, Lychee, Mango, Orange, Pear, Pineapple, Raspberry, and Strawberry). Finally, for shape, we compared ratings to rounded, spiky, pyramidal, and cube shapes.

2.2 Participants

A total of 86 participants were recruited via the Prolific online participant recruitment platform (https://www.prolific.com/) and passed attention checks (45 biologically female; M = 34.0 years, SD = 7.2 years, range = 22–50 years). Pre-selection criteria ensured all participants resided in the UK, spoke English as their first language, and were not colourblind, as these demographics expected to be the most prevalent for the subsequent in-person study. All participants completed the survey online using the Qualtrics online survey creation tool (https://www.qualtrics.com/uk/).

2.3 Results

2.3.1 Colour

For associations between colour and taste, two repeated measures ANOVAs were conducted, comparing sweetness and sourness ratings among each colour category (six groups). The analysis of sweetness revealed a significant main effect of colour with a large effect size (p < 0.001, ${\eta }_{\text{p}}^2$ = 0.19). Post-hoc t-tests with Holm correction for multiple comparisons indicated that orange was rated as significantly sweeter than all other tested colours (all p_Holm < 0.007, Cohen's d > 0.42). Red, yellow, and purple were not rated significantly differently from each other (p_Holm > 0.999, d < 0.15), but were all rated significantly sweeter than Green (p_Holm < 0.001, d > 0.55) and Blue at medium effect sizes (p_Holm < 0.001, d >0.51). These results are more clearly demonstrated in Figure 1, left. The analysis was repeated with sourness as the outcome variable. Again, there was a significant main effect of colour (p < 0.001, ${\eta }_{\text{p}}^2$ = 0.16), with Yellow and Green having the highest sourness scores than the remaining four colours (p_Holm < 0.006, d > 0.45), with the remaining four colours themselves not being significantly different from one another (p_Holm > 0.261, d < 0.269, Figure 1, left). From these results, yellow was not selected due to its associations with both sweetness and sourness. When selecting a “sweet” colour, red was favoured over orange and purple due to a larger variety of red-coloured fruit flavour choices. Green was selected to represent sourness, as it was the only colour exclusively associated with sourness.

FIGURE 1

Figure 1. Sweetness and Sourness Ratings from the Pre-Study. Each box with a different numerical value (1, 2, 3, or 4) signifies significant differences between levels based on t-test comparisons (p < 0.05). For instance, orange (1) was perceived as sweeter than red (2), yellow (2), and purple (2), which were all in turn sweeter than green (3) and blue (3). Left: Text icons represent colours (e.g., R = Red). Red was exclusively associated with sweetness, and green solely with sourness. Middle: Icons denote shapes (e.g., □ = square shape). Rounded shapes were linked to sweetness, and spiky shapes to sourness. Both cube and pyramid shapes were equivalently unrelated to sweetness and sourness. Right: Icons symbolise the tested flavours. Strawberry, cherry, and raspberry were strongly associated with sweetness compared to lime, which had the strongest association with sourness.

2.3.2 Flavour

Next, out of the 14 different fruit flavours tested, only those originating from red or green fruit were analysed (strawberry, raspberry, cherry, and lime). For sweetness, a repeated-measures ANOVA revealed a significant effect of flavour (p < 0.001, ${\eta }_{\text{p}}^2$ = 0.71). Briefly, strawberry was rated as sweeter than all other flavours (all p_Holm < 0.001, d > 0.44). Cherry was the second sweetest (all p_Holm < 0.012, d > 0.31), followed by raspberry which was sweeter than lime (p_Holm < 0.001, d = 2.0). For sourness, there was a significant effect of flavour (p < 0.001, ${\eta }_{\text{p}}^2$ = 0.55). Lime was rated as sourer than all other flavours (all p_Holm < 0.001, d > 1.20). Raspberry was rated as sourer than cherry and strawberry (both p < 0.043, d > 0.26), and cherry was sourer than strawberry (p < 0.001, d = 0.68). Overall, lime was an obvious choice due to its association with sourness. Strawberry was chosen as the second fruit, as it had the highest sweetness ratings and lower sourness ratings than the other three red fruits. Consequently, the crossmodal correspondences chosen were Red-Strawberry and Green-Lime, which correspond both in their colour and their associations with sweetness and sourness, respectively.

2.3.3 Shape

A final repeated measures ANOVA compared taste ratings between different shapes (Rounded, Spiky, Cube, or Pyramidal). There was a significant main effect of shape on sweetness ratings (p < 0.001, ${\eta }_{\text{p}}^2$ = 0.36), primarily due to the rounded shape being rated as significantly sweeter than the other shapes (p_Holm < 0.001, d > 1.2). The remaining shapes were not significantly different from one another (p_Holm > 0.181, d < 0.181). There was also a significant main effect of shape on sourness ratings (p < 0.001, ${\eta }_{\text{p}}^2$ = 0.22), with spiky shapes being considered sourer than all other tested shapes (p_Holm < 0.001, d > 0.699). The remaining shapes did not significantly differ from each other (p_Holm > 0.081, d < 0.292). The cube and pyramid shapes were equally unrelated to sweetness and sourness. The cube shape was chosen over the pyramid shape due to the simplicity of manufacturing and being more practical for handling in VR.

3 Main study

Our pre-study successfully identified Red-Sweet and Green-Sour as suitable colour-Taste associations, strawberry and lime flavours as appropriate crossmodal correspondences of these associations, and highlighted cube-shaped food as not associated to sweetness or sourness (e.g., to avoid shape biasing results). The main study aimed to assess the influence of ambient lighting on taste and flavour perception, and whether this effect interacted with the actual flavour of the taste samples. Taste samples in the current study were pieces of relatively firm jelly, which we refer to as jelly samples here after (see Table 1 for details). Participants were asked to eat three different flavours of samples (Lime, Strawberry, No Added flavour) under three different ambient lighting conditions (Green, Red, and Neutral, see Figure 2). It was hypothesised that:

1. samples eaten under red lighting would (a) taste sweeter and (b) be perceived more intensely of strawberry than those eaten under neutral and green lighting.

2. samples eaten under green lighting would (a) taste sourer and (b) be perceived more intensely of lime than those eaten under neutral and red lighting.

3. when there is a crossmodal correspondence between ambient lighting colour and sample flavour (e.g., Red-Strawberry) the effect on sample ratings will be stronger relative to when there is no crossmodal correspondence (e.g., Red-No added flavour).

TABLE 1

Table 1. Recipes used in the study.

FIGURE 2

Figure 2. Screenshot of the three lighting environments of red, neutral, and green light. The lighting was a blend of interior spotlights for red (RGB: 255:0:0) and green (RGB: 0:255:0) and neutral environments (RGB: 255:255: 255) with external digital sunlight through the windows.

3.1 Study design

The main study was a within-participants user study, conducted using a 3 (neutral, red, and green) × 3 (No added flavour, Strawberry, and Lime) experimental design. The selection of both ambient lighting colours and flavours for this study was based on the results of the online pre-study (see Section 2 above). Taste was evaluated through participants' ratings of sweetness and sourness, while flavour was assessed through ratings of the intensity of strawberry and lime flavour in the samples. Hedonic experiences were captured by evaluating the perceived enjoyment of the samples and the overall experience.

3.2 Participants

For the main study, 34 individuals were included (30 cis-gendered females and 4 cis-gendered males; M = 26.7 years, SD = 9.5 years, range = 18–60 years, all with normal or corrected-to-normal vision, an intact sense of smell and taste and were not vegan or vegetarian due to pork gelatine in the recipe). Participants were diverse in nationality, with the most common being Chinese (12), followed by British/British dual national (8), Western European (5), American (3), Southeast Asian (3), Eastern European (2), and finally individuals from Russia and New Zealand. All participants were requested to abstain from consuming spicy food for 24 h, any food or drink (except water) for 1 h, and alcohol for 6 h before participation, following prior research (Obrist et al., 2014).

3.3 Development of stimuli: flavoured jelly samples

After the study design process, we began to iteratively develop pleasant, flavoured jelly samples. We chose jelly as the target medium due to its associations with both sweetness and sourness, different fruit flavours, and to build on our previous work (Cornelio et al., 2022). Below, we describe the fabrication process of cube-shaped moulds and the development of the recipe.

3.3.1 Custom moulds

Custom silicone moulds, designed to produce 1.6 cm perimeter cubes (4.1 ml volume), were manufactured (see details in Figure 3). This was based on the sample volume of our previous study, so that we could infer a pleasant number of food samples participants could eat from past results. This process involved creating 3D printing negatives of the cube design from the food-safe filament. Upon completion, the moulds were washed, and food-safe silicone rubber (30A Platinum RTV-2 Liquid) was poured into the negatives. After 12 h, the silicone was removed from the negatives, cleaned, and used for sample production. The final silicone moulds were rigid enough to maintain their shape when heated but flexible enough to facilitate removal.

FIGURE 3

Figure 3. Jelly taste samples used in the study. Left: we used custom silicone moulds of cubes measuring 1.6 cm × 1.6 cm × 1.6 cm. Middle: there were three different flavours used, lime, unflavoured, and strawberry. Right: the jellies has a texture in between typical dessert jelly and gummy bears.

3.3.2 Recipe development

The primary goal of recipe development was to create pleasant lime, strawberry, and neutral recipes that remained enjoyable after 27 trials. Various recipes, inspired by our previous study (Cornelio et al., 2022), were created. An overview of the development process justifying the final recipe is provided below and is concisely summarised in Table 1. Development commenced with a base recipe comprising 100 ml of water and 9.6 g of pork gelatine, selected for its neutral flavour and its ability to provide structure. We then aimed to balance the perceived intensity of sourness with sweetness by adding varying quantities of citric acid (0.4g−1g) to 25 g of sugar. Strawberry and lime samples were initially prepared by adding 12 drops of Strawberry or Lime MyAroma natural flavouring to these recipes. The 12 drops were chosen to strike a balance between clear flavour identification and a pleasant eating experience. These initial recipes were then pilot tested by internal staff and seven participants. Feedback suggested that 0.5 g of citric acid was a good perceptual balance relative to 25 g of sugar and that higher intensities of citric acid were sometimes mistaken for the flavour of lime. However, the flavours of strawberry and lime were not perceived as intense by some participants, possibly due to a relatively high gelatine content. Consequently, we varied the gelatine content between 12.8 g and 6.4 g, with 8.1 g emerging as the minimum content that ensured a rigid sample at room temperature. To further distinguish the sour taste and lime flavour, two changes were made. First, flavours were enhanced by adding Monin flavoured syrups to the samples. The sugar content of these syrups was then deducted from the recipe so the total sugar remained around 25 g. Further internal testing (n = 13) suggested the final recipe was well-received, provided an enjoyable texture, and the flavours were clear. The second change involved adding a “taste training” session into the study to help users distinguish the tastes and flavours. These involved participants tasting four different samples, each containing 8.1 g gelatine per 100 ml: sweet only, sour only, strawberry only, and lime only. These recipes were created so a single taste/flavour was dominant with the minimum amount of sugar needed to counteract the taste of gelatine. These final recipes are presented in Table 1. We refer to the neutral control sample hereafter as “No added flavour.” The lime, no added flavour, and strawberry jelly samples along with the moulds used are shown in Figure 3.

3.3.3 Sample preparation

The cooking process began by blooming the gelatine (soaking in cold water) for 5 min in a saucepan, during which time the sugar was added and stirred, if required. After 5 min, the mixture was heated on low until the gelatine and sugar had completely dissolved (approx. 3 min). The mixture was then removed from the heat, and the citric acid, flavoured syrup, and flavour drops were added as necessary (see Table 1). The mixture was stirred until well incorporated. The mixture was then transferred into the silicone moulds using a syringe to prevent air bubbles. All samples were chilled overnight at 4 degrees Celsius and then removed from the moulds. All samples were brought to room temperature before serving to enhance the intensity of taste and flavour (approx. 20 mins out of the fridge). To minimise variation, all participants received samples prepared the same week and from the same batch. Furthermore, all samples were prepared by the same researcher. In the main experiment, each participant consumed 9 samples in each of the three different lighting environments (e.g., red lighting condition: 3 no added flavour, 3 strawberry, and 3 lime), resulting in a total of 27 samples. The training samples were created using the same process and participants consumed one of each type (i.e., one sweet only, one sour only, one strawberry only, and one lime only) before the main study.

3.4 Study setup and technology

The virtual environment was designed using the Unity Game engine and presented via a Meta Quest Pro headset, providing a resolution of 1800 × 1920 pixels per eye at a 90 Hz refresh rate. The real-world setting was a kitchen-style room equipped with a fridge, sink, cabinets, and seating for the participants and the researcher. In the virtual environment, we included counterparts of the real-world furniture, including the chair and desk at which the participants were seated. Participants always saw the jelly samples as grey cubes in the VR environment to remove the influence of colour (see Figure 5, middle). This ensured the virtual and real locations were congruent, allowing participants to accurately orient themselves and interact with the samples on the plate. The Meta Quest Pro's in-built hand-tracking technology facilitated these interactions.

3.5 Design of the virtual environment and ambient lighting colours

To evaluate the impact of ambient lighting colour, we used a realistic and neutral kitchen-living room environment (identical to Cornelio et al., 2022) that could be illuminated with neutral, red, or green lighting (see Figure 3). The virtual environment featured a breakfast bar where participants were seated, offering views of a sofa, TV, rug, bookshelf, coffee table, and other decorative elements. A modern kitchen was situated behind the participants. To minimise the influence of ambient noise from the testing environment, a neutral background noise (wind rustling through grass) was played at a low volume. The use of noise cancellation was not feasible as the researcher needed to communicate with the participants initially. A virtual model of the cube-shaped jelly was also presented on the virtual plate, appearing as an opaque white colour. One of the key improvements over our previous study was the increased realism of the virtual environment, including enhanced lighting, shadows, and reflections provided by the High-Definition Render Pipeline of Unity, which uses physics-based lighting.

3.6 Measures

A combination of different measurements and methods were used as part of the study. These ranged from pre-task demographics, in VR ratings of taste, flavour, and affective responses, and post-VR assessments of the overall experience.

3.6.1 Pre-task questionnaire

Demographic information including sex, gender, age, and nationality were collected. We also collected information on participants' taste preferences for the five basic tastes and their familiarity with Virtual Reality on a scale of 0% (not at all) to 100% (completely).

3.6.2 Taste, flavour, and affective assessment

To assess participants' ratings of sample sweetness and sourness as well as the intensity of strawberry or lime flavour in VR, a Visual Analogue Scale was used (VAS, Crichton, 2001). This scale consisted of a continuum on which participants could rate, for example, the sweetness of a sample on a continuous slider from 0 to 100. Only the labels “0” and “100” were presented at each extreme of the scale to reduce the amount of visual information while in VR. Participants could view their current numeric rating as they moved the virtual response sliders. In addition to taste and flavour perception, we asked participants how much they enjoyed both the samples and the general experience. These items were rated on the same scale as the taste assessment items.

3.6.3 Sense of presence assessment

To assess the Sense of Presence (the feeling of “being there”) reported by participants during the study, we used a presence questionnaire proposed by Witmer and Singer (1998). This consisted of 10 questions about the experience immersion and the total score was taken as the outcome variable. The same 0–100 scale of the taste attributes questions was used to be consistent and reduce the complexity of the task. See Supplementary material for all the questions.

3.6.4 Post-task feedback

After the main VR task, further questions were asked, and feedback was requested. Participants were asked to rate the six colours of the pre-study on the five basic tastes, to validate if the study participants had the same crossmodal correspondences between colour and taste as those in the pre-study. Further validation of the study aims was conducted by asking participants to rate the naturalness of the three coloured environments on a scale of 0 (Artificial) to 100 (Natural) with the image in Figure 2 used for reference. Participants were then asked to what extent they found eating in VR to be a) comfortable b) pleasant and c) enjoyable on a scale from 0 (“Not at all”) to 100 (“Very Much”) and whether they thought the taste experience would be perceived differently in real life relative to VR on a scale of 0 (“I disagree”) to 100 (“I completely agree”).

3.7 Study procedure

As illustrated in Figure 4, after passing the pre-screening criteria and consenting to their participation, participants took part in “taste training” (explained below). From our initial testing of the main study, participants found the distinction between lime flavour and sourness difficult in previous versions of the recipe. In addition to changes to the recipe, the taste training session was added so that participants could experience these tastes/flavours separately. This process also familiarised participants with the eating process and rating scales later used in VR.

FIGURE 4

Figure 4. Experiment procedure followed by participants outside of VR activities.

Participants ate a sweet, strawberry, sour, and then lime sample sequentially and sipped water in between. This order was chosen so participants could immediately compare the more closely related tastes (e.g., Sweet -> Strawberry, Sour -> Lime). Participants rated these samples on the taste, flavour, and affective items outlined earlier (see Section 3.6.2) so that any question clarity issues could be resolved. Only one repetition of this training was given not to exhaust participants with excessive samples, considering the main study involved 27 samples. Participants saw these jelly samples, with the lime and strawberry being slightly green and red respectively. However, it was made clear to participants the samples they would eat in VR would not be the same as those eaten in this practise.

After participants confirmed they were happy to continue, they next completed the demographic information before continuing to the VR task (explained in detail in the next section). The VR task was followed by the post-task feedback and debrief. The study took on average 75 min to complete and each participant was given a £15 voucher to thank them for their participation.

3.8 Virtual reality task

The VR task contained two primary portions: a practise and the main tasting study. The former aimed to familiarise users with the technology before the primary study began, as outlined below.

3.8.1 Explanation and practise

IRL, participants sat at the breakfast bar-style table facing the blue wall. When entering VR, they saw a replica IRL kitchen-style environment including the table and chair. The practise began with a notice board presenting task instructions. The first mechanic explained was gesturing. Participants were asked to hold a thumbs-up gesture with both hands for 500 ms to signal “OK,” “Next,” or “I'm ready” (Figure 5, left). When a gesture was recognised, the task progressed, and a sound effect (a positive sounding “ding”) was played. The next mechanic to practise was grabbing, which consisted of pinching with the right thumb and index finger to either grab jelly samples or respond using the sliders. During this practise, the experimenter placed a jelly sample on the plate and spawned the respective virtual 3D model in the same location (shown as a grey cube). Importantly, participants never saw the three IRL jelly samples, only the digital representation. Participants were then asked to grab this sample and place it onto a second plate in front of them (Figure 5, middle). Finally, participants practised grabbing the sliders by pinching a centre tick mark and moving it along a slider (Figure 5, right). Participants practised each mechanic three times and then verbally confirmed comprehension.

FIGURE 5

Figure 5. Gesturing mechanics explained during the practise phase. Left: thumbs up gesture to progress during the VR task, Middle: pinching gesture to either grab the taste stimulus or Right: use rating using the scales. The VR representation of the jelly sample is highlighted in the red circle.

3.9 Tasting in VR

As shown in Figure 3, after the practise, participants were teleported into one of the three rooms at random (i.e., neutral, red, or green) with the location of the table, chair, and plate being persistent. Participants used the learnt gestures to progress through the instructions. In each room, participants completed a brief visual search task, consisting of finding two objects (a plant and a photo frame) and rating the object's appealingness using the sliders. This was done to ensure participants looked around the room and did not isolate their attention to the table alone. After participants confirmed their ratings, the primary task instructions were presented.

Within the experimental trial loop (see “taste” in Figure 6), each trial began with the experimenter placing a jelly sample on the plate and spawning a digital copy in VR. The order of jelly samples was randomised for each lighting condition. Participants then took the sample with their right hand and put it in their mouth. The notice board asked participants to explore the shape in their mouth and counted down from 5 seconds (Figure 7). Next, participants were told to chew and swallow the sample. This information was presented for 8 seconds to ensure participants took their time to explore the taste/flavour. Next, the notice board informed participants that they should rate their experience using the sliders (Figure 5), which animated towards the participants. The taste, flavour, and affective questions were presented in blocks of which the order was randomised.

FIGURE 6

Figure 6. Experimental procedure followed by participants during the VR phase which consisted of 27 trials (samples).

FIGURE 7

Figure 7. Task flow diagram of each trial. This procedure was repeated 9 times in each of the three ambient lighting conditions (neutral, red, green) resulting in a total of 27 trials (samples).

After all questions were answered participants confirmed their choices with gesturing, the sliders animated to the side, and the notice board asked participants to wait for the next sample. This procedure was repeated 9 times in total for each of the three ambient lighting conditions (neutral, red, green) resulting in a total of 27 samples being eaten. Between lighting conditions, the SoP questionnaire was completed in VR and there was a 2-min break in which participants took the headset off and had a sip of water. When participants re-entered the environment, they were in the kitchen-style room again and the process was repeated. When all 27 samples were eaten (40–50 min) participants then rated their overall experience of the VR environment on Qualtrics.

3.10 Data preparation and analysis strategy

After the removal of 7 pilot participants who experienced different variations of the recipe, the 34 participants who experienced the final recipe were included in the analysis. The first analysis aimed to validate that the crossmodal correspondences expected from the pre-study were replicated in the main study. This involved conducting two non-parametric within-samples t-tests (Wilcoxon Signed-Rank tests) comparing taste expectations (not actual experiences during the study) of sweetness and sourness between red and green lighting. A further validation analysis was conducted to ensure the flavours of the jelly samples were distinct, involving non-parametric repeated-measures ANOVAs (Friedman's tests) assessing either strawberry or lime flavour intensity between each sample flavour. We also then explored the Sense of Presence (SoP) of participants in each environment and their ratings of each environment's naturalness using the same statistical procedure. The main effects of these analyses were all Bonferroni corrected for multiple comparisons to avoid inflated Type I error rates (α = 0.05/5 comparison = 0.01). Finally, to assess whether taste or flavour habituation had occurred (i.e., too many samples may have made participant's insensitive to detect the differences from over-stimulation, which may explain a failure to replicate crossmodal correspondences), a beta Linear Mixed Effects (LME, explained below for the main analysis) model was conducted comparing ratings of taste, flavour, and affect in the first, second, and third block (room) of nine trials. Comparisons between blocks were also Bonferroni corrected for multiple comparison.

The primary analyses investigated (1) whether ambient lighting colour affected ratings of sweetness and sourness across all samples and (2) whether a crossmodal correspondence between ambient colour and sample flavour augmented ratings (e.g., Red–Strawberry increasing sweetness rating relative to Red-No added flavour). To achieve this, the outcome variables were analyzed using Linear Mixed-Effects models (LME, Pinheiro and Bates, 2000) which are similar in interpretation to linear regression. Each model contained the random-effects of trial repetition and participant and the fixed-effects (“main effect”) of sample flavour and ambient lighting colour. As data distribution appeared bi-modal, a beta rather than Gaussian distribution was used in the analysis, after which model parameters were back-transformed to the original units for interpretation. Crossmodal correspondences were investigated through the interaction effects between colour and flavour. All analyses were conducted in R studio (R Development Core Team, 2019) using several statistical (Lenth et al., 2020) and data visualisation (Wickham, 2016) packages.

4 Results

4.1 Recipe validation

Both lime flavour intensity and strawberry flavour intensity significantly differed across recipes (both p_Bonf < 0.001). Planned comparisons confirmed that the lime samples were more intensely flavoured of lime (median = 71.2%, Median Absolute Deviation [MAD] = 11.6) than both the no added flavour samples (median = 7.6%, MAD = 8.56) and strawberry samples (median = 10.8%, MAD = 12.1) at a large effect size (p < 0.001, both r_{rank−biseral} = 1.000). Similarly, the strawberry samples were more intensely flavoured of strawberry (median = 59.4%, MAD = 18.6) than both the no added flavour (median = 21.8%, MAD = 13.8) and lime samples (median = 7.56%, MAD = 10.0) both at large effect sizes (p_Bonf < 0.001, r_{rank−biseral} > 0.990). Together, this validates the manipulation of flavour was successful.

4.2 Crossmodal correspondence expectations

Participants suggested the colour red was significantly more associated with the taste of sweetness than the colour green (p_Bonf < 0.001, r_{rank−biseral} = 0.780, Figure 8, right). Similarly, participants rated the colour green as significantly more associated with sourness than red (p_Bonf < 0.001, r_{rank−biseral} = 0.862).

FIGURE 8

Figure 8. Participant ratings of their Sense of Presence (SoP, left), the naturalness of lighting (middle) between the neutral, green, and red rooms, and their associations between colour and taste (right). SoP did not differ between rooms, but the neutral room was rated as much more natural. Participants associated red more with sweetness than they did with green, and associated green more with sourness than red. ***p_Bonf < 0.001.

4.3 Sense of presence and naturalness of lighting

Ratings of Sense of Presence (SoP) did not significantly differ between each room (χ²(2) = 4.3, p_Bonf =0.205, Figure 8, left). However, for naturalness, there was a significant main effect of colour at a large effect size [${\chi }_{(2)}^2$ = 49.8, p_Bonf < 0.001], due to the neutral environment being rated as significantly more natural (median = 91.0%, MAD = 12.6) than both the red environment (median = 19.5%, MAD = 23.7) and green environment (median = 27.5%, MAD = 29.7) at a large effect sizes (both p_Bonf < 0.001, r_{rank−biseral} > 0.993), although the red and green environments did not significantly differ (p_Bonf = 0.886, r_{rank−biseral} = 0.255).

4.4 Taste and flavour habitation and changes in affect experience over time

Taste and flavour ratings were predicted from the fixed-effect of room order and the random-effect of flavour, repetition, and participant. There was trend-level evidence that ratings of sweetness reduced from the first room of nine trials (i.e., the first VR environment depending on the counterbalanced order) to the second room of nine trials (p_Bonf = 0.074). However, the ratings were significantly reduced in the third room compared to the first (p_Bonf < 0.001), potentially suggesting taste habituation to sweetness occurred after 18 samples. Comparisons for the variables of sourness and both strawberry and lime intensity were non-significant (Figure 9), suggesting habituation did not occur. For affective experiences, participants rated samples as less enjoyable in the second room compared to the first (p_Bonf < 0.001) which was also then lower in the third room compared to the second (p_Bonf < 0.001). A similar pattern was shown for enjoyment of the experience, with trend-level evidence of a decline in the second room (p_Bonf= 0.082) which thereafter significantly declined in the third room (p_Bonf < 0.001). This suggests that participants began to like the experience less after nine samples (approx. 10 min).

FIGURE 9

Figure 9. Analysis of taste and flavour perception over time. Half-violins represent the data distributions and point ranges represent the back-transformed model coefficients from the beta LME models and the 95% Confidence Intervals [CIs]. There was trend-level evidence for lower sweetness ratings in the second room compared to the first, with significant evidence of reduced ratings in the third room compared to the first. In terms of affective experience, participants rated the samples as less enjoyable and enjoyed the overall experiences less as the task progressed. ^†p_Bonf < 0.01, *p_Bonf < 0.05, ***p_Bonf < 0.001.

4.5 Virtual reality appraisal

The descriptive statistics of previous VR experience and post-VR appraisals can be found in Figure 10. Overall, participants rated the experience as comfortable (median = 77%), enjoyable (70%), and pleasant (75%). Participants generally felt that the taste experience would be perceived differently in real-life (65%). Finally, participants came from a wide range of backgrounds in terms of VR familiarity (39%), from first-time users to consistent users.

FIGURE 10

Figure 10. Descriptive statistics of participant ratings of their experience of eating in VR and previous experience with VR in general before the study. Perceived Different = the tasting experience would be perceived differently in real life compared to VR.

4.6 The effect of ambient lighting on taste and flavour

To test our main hypotheses, we assessed the influence of ambient lighting colour on taste, flavour, and affective experiences. We also assessed whether the potential effects of colour would be stronger if the actual sample flavour was congruent with the colour (e.g., Red-Strawberry) relative to being incongruent with colour (e.g., Red-No added flavour).

4.6.1 Colour and taste

The primary analyses began with investigating the effect of ambient lighting on the taste variables of sweetness and sourness, as well as any interaction with sample flavour. Starting with sweetness, a model was specified with the fixed-effects of colour and flavour and the random-effects of repetition and participant (AIC: −749, R${}_{\text{m}}^2$ = 16.9¹). There was a marginally significant effect of red lighting to increase sweetness ratings of samples relative to neutral lighting (z = 1.975, p = 0.048), although the effect of green lighting was non-significant (z = 1.068, p = 0.285). Next, a further model was specified with an interaction term between colour and flavour. A significant result in this model would indicate that the effect of colour depends on the sample flavour. For instance, the effect of red lighting might be stronger in strawberry samples compared to neutral samples. However, this model was not a better fit to the data and thus this possibility was not supported [${\chi }_{(4)}^2$ = 1.879, p = 0.758, AIC: −743, R${}_{\text{m}}^2$ = 16.9]. For sourness, the process was repeated, with a model again including fixed-effects of colour and flavour with random-effects of repetition and participant (AIC: −1,781.3, R${}_{\text{m}}^2$ = 51.8%). However, neither red nor green lighting showed significant effects on sourness (red: z = −0.898, p = 0.369; green: z = −0.281, p = 0.779). The interaction model was not a better fit to the data [${\chi }_{(4)}^2$ = 2.3116, p = 0.679, AIC: −1,775.6, R${}_{\text{m}}^2$ = 51.8%] and the interaction effects were not statistically significant (Figure 11).

FIGURE 11

Figure 11. Plotted results of beta Linear Mixed-Models predicting sample sweetness (left) and sample sourness (right) in each of the three different ambient lighting environments. There was a significant effect of red lighting to increase sample sweetness compared to neutral lighting. For each graph, left-side point range plots represent the predicted coefficients of the model and 95% Confidence Intervals, whereas the right-side violin plots represent the raw data. ^*p < 0.05.

4.6.2 Colour and flavour

For strawberry flavour intensity, the effect of both red (p = 0.655) and green lighting was non-significant (p = 0.673, AIC: −1,466.4, R${}_{\text{m}}^2$ = 51.3%, Figure 12, left). The subsequent interaction model was not a significantly better fit to the data [${\chi }_{(4)}^2$ = 6.8682, p = 0.143, AIC: −1,465.3, R${}_{\text{m}}^2$ = 51.5%], although there was a marginally significant interaction between green lighting and lime flavour (p = 0.047). However, post-hoc t-tests investigating this interaction were non-significant. This was also the case for a trend-level interaction between red lighting and lime flavour (p = 0.073). Together with the non-significant chi-square test, these marginal effects should be interpreted cautiously. For lime flavour intensity, both green (z = 1.288, p = 0.198) and red lighting showed no significant effects (z = 0.864, p = 0.388, AIC: −2,051.8, R${}_{\text{m}}^2$ = 65.7%). The subsequent model including the interaction term was not a significantly better fit to the data [${\chi }_{(4)}^2$ = 1.41, p = 0.843, AIC: −2,045.2, R${}_{\text{m}}^2$ = 65.7%), and none of the interaction terms was found to be significant (p > 0.844). Overall, there was no evidence for an effect of ambient lighting on flavour perception.

FIGURE 12

Figure 12. Plotted results of beta Linear Mixed-Models predicting strawberry flavour intensity (left) and lime flavour intensity (right) in each of the three different ambient lighting environments. No comparison was significant. For each graph, left-side point range plots represent the predicted coefficients of the model and 95% Confidence Intervals, whereas the right-side violin plots represent the raw data.

4.6.3 Colour and affective experience

For sample enjoyment, the effect of lighting was non-significant for both red (z = 0.693, p = 0.488) and green lighting (z = −0.394, p = 0.693, AIC −1,107.5, R${}_{\text{m}}^2$ = 7.7%). The subsequent model interaction model was not a significantly better fit to the data [${\chi }_{(4)}^2$ = 0.737, p = 0.947, AIC = −1,100.3, R${}_{\text{m}}^2$ = 7.75%] and all interaction terms were non-significant (p > 0.489, Figure 13). For experience enjoyment, both green (z = −1.181, p = 0.238) and red lighting (z = −0.762, p = 0.446) showed no significant effects (AIC: −1,470.7, R${}_{\text{m}}^2$ = 3.56%). The subsequent model including the interaction term was not a significantly better fit to the data [${\chi }_{(4)}^2$ = 1.371, p = 0.849] and none of the interaction terms was found to be significant (p > 0.478).

FIGURE 13

Figure 13. Plotted results of beta Linear Mixed-Models predicting sample enjoyment (left) and experience enjoyment (right) in each of the three different ambient lighting environments. No comparison was significant. For each graph, left-side point range plots represent the predicted coefficients of the model and 95% Confidence Intervals, whereas the right-side violin plots represent the raw data.

4.7 Previous virtual reality experience and taste experience

To further explore reasons why IRL crossmodal research may not apply to XR scenarios, potential moderators of these effects were assessed. In our previous study (Cornelio et al., 2022) participants stated in open-ended comments that the unusualness of the experience of eating in VR affected their taste perception. As a result, we added VR familiarity as a potential moderating variable in the previous analyses. While there was evidence that increased VR familiarity decreased sweetness ratings in the neutral room relative to the red room, we add this analysis to the Supplementary material. We found inconsistent patterns with sourness, strawberry, and lime, and feel this inconsistency means the significant effect for sweetness should be interpreted cautiously. We add this here for transparency and provide further details in Supplementary Figure 1.

5 Discussion

The current research aimed to investigate whether crossmodal correspondences between colour and taste found In Real Life (IRL) also apply to considering research is highly mixed on this topic. We suggested that a lack of transferal may be the limited realism of some XR scenarios, which this study aimed to overcome through a high-fidelity scenario. To investigate this, we conducted an online pre-study to select crossmodal correspondences of sweetness and sourness and an in-person study on taste experiences with these selected variables. The pre-study supported Red-Sweet and Green-Sour as strong crossmodal associations, strawberry and lime as flavour associations to Red-Sweet and Green-Sour, and cube-shaped samples as a suitable control sample due to equal associations to sweet and sour. The main study, therefore, assessed crossmodal correspondences between ambient lighting colour (red, green, neutral) on perceived taste (sweet/sour) and perceived flavour (strawberry/lime), as well as whether there was an interaction between colour and the actual flavour of the samples (no added flavour/strawberry/lime). Validation analyses confirmed the three sample types were distinct in flavour, that participants associated red colour with sweetness and green colour with sourness, and that participants were equally immersed in the three ambient lighting environments. However, the red and green environments were rated as vastly less natural than the neutral lighting environment and taste habituation to sweetness occurred. The main results suggested there was a marginally significant effect (p = 0.048) of red ambient lighting to increase ratings of sweetness relative to neutral lighting. However, there were no differences in ratings of sourness nor the intensity of strawberry or lime flavour between any lighting conditions. Moreover, there were no significant interactions between ambient lighting colour and actual sample flavour, suggesting that the effect of red lighting on sweetness was not dependent on sample flavour. Finally, it was suggested that people who were more familiar with VR found samples eaten under the neutral lighting to be less sweet than those eaten under the red lighting. Overall, there was minimal evidence of multisensory integration in XR. Below, we first outline how the current results fit with our hypotheses and within past literature before outlining potential explanations behind these findings.

5.1 Crossmodal associations between colours and taste/flavour

The results of our pre-study are closely aligned with past research. For colour, the common association between red and orange with sweetness was replicated (Strugnell, 1997; Saluja and Stevenson, 2018). Yellow presents an interesting finding wherein it is associated with both sweetness and sourness, likely due to fruits such as bananas/lemons. Another expected finding was that green was associated with sourness (Koch and Koch, 2003; Tomasik-KrÓtki and Strojny, 2008; Wan et al., 2014; Huisman et al., 2016; Saluja and Stevenson, 2018). Past literature has also sometimes associated green with bitterness (O'Mahony, 1983; Spence and Levitan, 2021; Spence, 2023) with inconsistencies potentially due to cross-cultural differences and confusion between sour and bitter (Levitan et al., 2014; Wan et al., 2014), blue did not appear related to sourness, although past research has also reported similar findings (Koch and Koch, 2003; Wan et al., 2014) and that blue appears related more so to saltiness (Tomasik-KrÓtki and Strojny, 2008; Spence and Levitan, 2021). For shape, we replicated that bouba-shaped samples are associated with sweetness while kiki-shaped samples are associated with sourness (Ngo et al., 2013; Huisman et al., 2016), whilst also suggesting both pyramidal and cubic shapes as suitable control shapes when investigating sweetness and sourness. Finally, we identified strawberry and lime as associates of sweetness and sourness and candidates for future research on flavour. In the main study, these Red-Sweet and Green-Sour expectations were replicated again, giving further strength to the results of the pre-study. These insights could prove valuable for researchers planning to conduct crossmodal studies among UK participants, as they provide a robust foundation and guide for experimental design.

However, despite these insights on taste expectations, our main study's findings only partially align with our expectations and prior research. Specifically, we found that red lighting enhanced the perception of sweetness (Hypothesis 1a), consistent with VR studies revealing an increase in sweetness in both pink environments (Wu et al., 2022) and sweet-congruent environments (Chen et al., 2020) as well as IRL studies (e.g., Velasco et al., 2013; Motoki et al., 2021). This result, however, contradicts both our previous study employing a similar procedure but at lower fidelity (Cornelio et al., 2022) and another past VR study (Nygård and Lie, 2020). It should also be noted that our pre-study selected colours in a quite a broad context (e.g., “to me, the colour red is sweet”), to which participants may have responded concerning the colour of food, rather than more general extrinsic factors (such as lighting). Moreover, it is critical to emphasise that the influence of red lighting was only marginally significant and should be interpreted cautiously (p = 0.048). A further unexpected finding was the lack of association between green and sourness ratings (Hypothesis 1b), although recent VR studies have reported similar results (Wu et al., 2022) as well as a lack of association with perceived sweetness ratings (Chen et al., 2020). Finally, no associations were found between ambient colour and flavour intensity (Hypotheses 2a & 2b). While IRL studies have found an influence of colour on flavour perception, such as green lighting increasing the “fruitiness” of citron vodka (Wang and Spence, 2015), there are limited assessments in XR. Those that have been conducted tend to focus on colour congruency in flavour identification rather than flavour intensity (Nishizawa et al., 2016; Ammann et al., 2020). Currently, no study has investigated the association between ambient colour and strawberry/lime flavour in VR environments, with this first study not supporting such a relationship. That being said, it is important to highlight that perhaps the influence of ambient lighting may be apparent with alternative descriptors. Specifically, while red may not increase strawberry flavour intensity, perhaps it could increase “fruitness”, “freshness”, or “smoothness”. Although, it has been suggested that taste and vision consider different kinds of sensory information.

The minimal impact of ambient colour on taste/flavour experiences is especially surprising, considering that the current participants stated they strongly associated Red-Sweet and Green-Sour (see Section 4.2); highlighting crossmodal correspondences in the expectation of taste but limited influence on the experience of taste in XR. Together with past research, there is only weak evidence to support an association between red/pink environments with elevated sweetness levels, which is far below that expected from IRL experiments. So, why might this lack of translation have occurred for both taste and flavour?

5.2 Why may crossmodal associations between colours and taste/flavour not transfer to digital environments?

We hypothesised that the reduced visual realism of colourful VR environments relative to the neutral environment may inhibit the effects of IRL taste associations transferring to XR. Although the current study was a significant improvement in terms of fidelity, the red and green environments were still judged as unnatural, despite environments only differing in the hue of interior spotlights (perhaps due to the infrequency of seeing such environments IRL). Consequently, environmental realism may have still played a role. Similarly, this may have been exacerbated by the artificialness of the digital sample and food interaction method. In real life, we physically interact with food—we touch it, feel its texture, smell its aroma, etc. In the current VR study, this physical interaction was limited to grabbing and users had little agency in their food experiences. Moreover, participants saw a grey cube to reflect the real jelly sample. While we aimed for this to act as a “neutral” representation, the grey cube may have been too far removed from real food. This is important as people report using their past (IRL) taste experiences as a justification for their colour-taste associations (Saluja and Stevenson, 2018). This is concordant with the crossmodal statistical account in which taste perception is influenced by natural correlations learned from prior experiences where we extract information from the properties of food (Spence, 2011). As a result, users may not have seen the scenario or digital food representation as food relevant, and therefore not apply their IRL taste expectations of Red-Sweet or Red-Strawberry and Green-Sour or Green-Lime to these digital experiences. In comparison, IRL lab studies do not have these limitations and even extend to public events (e.g., Velasco et al., 2013; Spence et al., 2014), which are more naturalistic and reflective of shared eating experiences. Creating truly realistic XR environments is a key consideration for future research. This is relevant not only for visual fidelity but also for the realism of interactions. Specifically, in the current study, food samples were presented to participants without any accompanying information. Perhaps future XR designs could integrate choice into the experience such as ordering from a menu, use clear representations of food as the digital artefact, being served food by an agent, using utensils, and creating a more applicable restaurant environment with other sensory cues consistent across conditions (e.g., atmospheric music, aromas of the food, tactile handing of samples etc.). Increasing contextual cues such as these may activate specific crossmodal correspondences sought after in our study. Alternatively, the continued emergence of truly mixed-reality technology may be a useful tool for future researchers. Specifically, information of real food in a real environment can be modulated with digital information in a realistic manner, perhaps overcoming the aforementioned limitations.

The realism of the food interaction scenario may have also affected users in another way, by altering their decision-making processes and weighting of sensory information relative to IRL studies. Specifically, eating in VR is a very novel experience which might create a sense of uncertainty or unfamiliarity for users, making it more difficult for them to apply their prior knowledge about food and taste. This uncertainty could disrupt the usual associations between sensory cues and taste expectations, as well as prevent users from combining prior knowledge and sensory information in a way that influences taste perception (Spence, 2011), leading to different taste perceptions in VR compared to IRL. Moreover, the current study was relatively cognitively demanding due to the eating mechanics to be learned. The effect of this may have been two-fold: focusing attention away from distal sensory experiences (such as the environmental colour) towards more task-centred information (task instructions) and putting greater focus on the task itself leading to greater attention to the real-world experience of taste. In these new scenarios, people may take a more critical and analytical approach to their responses and their sense. Users may feel anxious as they are unaware what to expect and they may feel they are “being tested”. This may mean that less focus is put on food extrinsic variables (e.g., ambient colour) and more focus is put on food intrinsic variables such as taste (Ammann et al., 2020), which would give participants more relevant information in the uncertain context. This may also be exacerbated by the reduced naturalism of eating in XR, wherein heuristic reasoning (such as crossmodal correspondences) may not be applied due to the new eating context. In contrast, users in IRL studies may be less analytical and anxious in assessing food samples, as this is an everyday task.

Indeed, some previous XR studies that do show an effect of environment type include environments with greater relevancy to food, such as a bar or winery (Picket and Dando, 2019), which may promote the application of IRL associations to XR environments. Perhaps the context of the environment in the current study, of an “at-home” eating experience, was too far removed from easily recognisable eating experiences. This may be particularly important for digital environments wherein users know the world is completely irrelevant to their real-world experience of taste, whereas the same environment IRL would hold more weight. For example, while the dark ambience of a restaurant is not directly relevant to taste IRL, it may be associated with skilled chefs, forming a realistic expectation of food quality. In comparison, the same ambience in VR is irrelevant according to users' learned beliefs (or lack thereof). As recommended above, increased contextual cues may be required to sufficiently activate learned IRL knowledge of food interactions.

In both IRL and VR studies, users' responses might be influenced by social desirability bias, where they give the responses, they believe are expected or desirable. Similarly, experimenter effects, where the presence or expectations of the researcher influence participants' behaviour, could also play a role. In the current context, users in both IRL and VR studies may guess that they are expected to change their ratings in accordance with ambient colour. Due to being observed users may be primed to respond in accordance with crossmodal correspondences or exaggerate ratings. However, in the current VR study, users were unaware that the experimenter can see their ratings and they were highly immersed in the digital environment (see Section 4.3), potentially leading to reduced biases from the real world that may result in smaller effect sizes as observed in the current results. Similarly, some past studies ask participants for their taste expectations before their taste experience scores, which has the potential to prime participant to answer following their expectations (Ueda et al., 2020), whereas the current study asked for expectations post-study.

Another potential explanation is the type of food sample used in the current study. The first consideration is the level of taste/flavour in the current samples. A series of studies by Wang et al. (2017) revealed that “spicy” music affected taste expectations of spiciness but not taste experiences when the actual spice level was mild, but did increase perceived spiciness when the spice level was moderate. In the current study, the levels of sweetness (M = 46.8%, SD = 28.3%) and sourness (M = 30.1%, SD = 28.8%) could be argued as being moderate and low respectively, which could explain why effects were seen for sweetness but not sourness. Although in our previous study the average sweetness rating of the “sweet” samples was 66.7%, we did not find an effect of red lighting (Cornelio et al., 2022).

A second consideration is the form of jelly sample we used. Although green has strong associations with sourness IRL (Huisman et al., 2016), it has a complex context dependent relationship with flavour (Shankar et al., 2010). Green colour is most associated with lime in terms of liquids and with vegetables in whole food form (Spence, 2023). Similarly, the colour of food can serve as an indicator of its caloric content. For instance, red-coloured foods are often perceived as having higher caloric content, while green foods, such as leafy vegetables, are typically associated with lower caloric content (Foroni et al., 2016). In terms of past research, both IRL (Velasco et al., 2013) and XR studies (Chen et al., 2020; Wu et al., 2022) have tended to focus on beverages such as wine or tea, which may have a stronger Red-Sweet/Green-Sour association than whole foods, such as the jelly-like samples in our current studies which both found minimal evidence of multisensory integration (Cornelio et al., 2022). It was intended that the jelly-like structure of the samples would elicit these associations, however, perhaps the digital grey cube model in VR inhibited this association. Indeed, it has been suggested that vision has dominance in influencing taste perception (Liu et al., 2019; Weidner et al., 2023). Therefore, our choice of grey cubed jelly samples may have inhibited the influence of ambient light on taste because this stimulus may not be congruent with prior food experiences. Future studies using such samples may wish to colour the samples as their respective flavour or use translucent digital modes. Similarly, modulating sample colour rather than ambient colour may lead to greater modulation of taste in XR.

5.3 Strengths and limitations

The primary strength of this study is that the experiment focused on actual taste experiences, rather than solely on taste expectations. Similarly, there is minimal information available on flavour intensity perception within XR, an area where this study provides valuable insights. Moreover, this study had a strong experimental design including multiple repetitions of sample rating, validation of taste expectations, and minimising experimenter effects using VR. However, several limitations should also be highlighted. Firstly, the sample size of the current study should be improved in future replications, perhaps by using a simpler experimental design. Secondly, while the sample was culturally diverse there was a clear dominance of females in the current sample. Indeed, pervious research has highlighted gender may influence the effect of ambient lighting on taste/flavour (Spence et al., 2014), and so these may only be generalisable to females. Thirdly, taste habituation to sweetness occurred after the first block of trials which may have dampened experimental effects. Finally, while the fidelity of the VR environments was high relative to previous work, the red and green environments were still rated as unnatural.

6 Conclusions

This study aimed to replicate crossmodal correspondences between ambient lighting colour and taste/flavour in XR, considering previous literature is currently mixed on whether results IRL apply to the digital world. We found limited evidence that lighting colour affects taste or flavour perception, beyond a small effect of red lighting to increase sweetness ratings, despite participants suggesting they strongly associated red with sweetness and green with sourness. We suggested several reasons for this observation, primarily focusing on the artificial nature of XR environments and food interactions which may inhibit learnt real world experiences from being applied. Moreover, we suggest that users may be more critical in XR food studies than in real world studies, which may lead them to focus on food-intrinsic elements (e.g., actual taste). There is a strong need for a richer understanding on these contrasting experiences to both understand multisensory integration and to design engaging multisensory taste experiences in the emerging metaverse. Future research may consider contrasting IRL and XR taste experiences within the same participants.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by the University College London's Ethics Committee Review Board. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

CD: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Visualization, Writing—original draft, Writing—review & editing. SB: Investigation, Methodology, Writing—original draft, Writing—review & editing. MS: Conceptualization, Methodology, Resources, Writing—review & editing. MM: Conceptualization, Methodology, Resources, Writing—review & editing. SP: Conceptualization, Methodology, Resources, Writing—review & editing. JZ: Conceptualization, Methodology, Resources, Writing—review & editing. GN: Methodology, Project administration, Resources, Supervision, Writing—review & editing. NK: Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Writing—review & editing. MO: Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Writing—review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was supported by Sony Europe B.V, ZN Deutschland (award No.: 156822). Recipient: MO.

Acknowledgments

The authors would like to thank members of the Multi-Sensory Devices (MSD) group at University College London for their feedback on the study setup and recipe design. Special thanks go to Giada Brianza for critical feedback on the crossmodal associations exploration and James Hardwick for his help in creating the moulds for the jelly samples. We like to thank the participants for taking part in our exploration of taste/flavour perception in VR.

Conflict of interest

MS, MM, SP, JZ, GN, and NK are employed by Sony Europe B.V. The authors declare that this research was conducted as part of a commercial and financial relationship between UCL and Sony. However, neither party benefitted financially from the outcome of this research.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcomp.2023.1268849/full#supplementary-material

Footnotes

1. ^R${}_{\text{m}}^2$ is the marginal R², which is analogous to the proportion of variance explained in the outcome variable due to the fixed-effects in the model.

References

Ammann, J., Stucki, M., and Siegrist, M. (2020). True colours: Advantages and challenges of virtual reality in a sensory science experiment on the influence of colour on flavour identification. Food Qual. Prefer. 86, 103998. doi: 10.1016/j.foodqual.2020.103998

CrossRef Full Text | Google Scholar

Brown, J., Zhu, M., Moran, M., Hoe, C., Frejas, F., and Cohen, J. E. (2021). “It has candy. You need to press on it”: Young adults' perceptions of flavoured cigarettes in the Philippines. Tob. Control 30, 293–298. doi: 10.1136/tobaccocontrol-2019-055524

PubMed Abstract | CrossRef Full Text | Google Scholar

Carvalho, F. M., and Spence, C. (2019). Cup colour influences consumers' expectations and experience on tasting specialty coffee. Food Qual. Prefer. 75, 157–169. doi: 10.1016/j.foodqual.2019.03.001

CrossRef Full Text | Google Scholar

Chen, Y., Huang, A. X., Faber, I., Makransky, G., and Perez-Cueto, F. J. A. (2020). Assessing the influence of visual-taste congruency on perceived sweetness and product liking in immersive VR. Foods 9, 465. doi: 10.3390/foods9040465

PubMed Abstract | CrossRef Full Text | Google Scholar

Cornelio, P., Dawes, C., Maggioni, E., Bernardo, F., Schwalk, M., Mai, M., et al. (2022). Virtually tasty: An investigation of the effect of ambient lighting and 3D-shaped taste stimuli on taste perception in virtual reality. Int. J. Gastron. Food Sci. 30, 100626. doi: 10.1016/j.ijgfs.2022.100626

CrossRef Full Text | Google Scholar

Crichton, N. (2001). Visual analogue scale (VAS). J. Clin. Nurs. 10, 706–706.

Google Scholar

Duffy, V. B., Hayes, J. E., Bartoshuk, L. M., and Snyder, D. J. (2009). “Taste: vertebrate psychophysics,” in Encyclopedia of Neuroscience (Elsevier). doi: 10.1016/B978-008045046-9.01674-0

CrossRef Full Text | Google Scholar

Etzi, R., Spence, C., Zampini, M., and Gallace, A. (2016). When sandpaper is “kiki” and satin is “bouba”: An exploration of the associations between words, emotional states, and the tactile attributes of everyday materials. Multisens. Res. 29, 133–155. doi: 10.1163/22134808-00002497

PubMed Abstract | CrossRef Full Text | Google Scholar

Fairhurst, M. T., Pritchard, D., Ospina, D., and Deroy, O. (2015). Bouba-Kiki in the plate: combining crossmodal correspondences to change flavour experience. Flavour 4, 22. doi: 10.1186/s13411-015-0032-2

CrossRef Full Text | Google Scholar

Foroni, F., Pergola, G., and Rumiati, R. I. (2016). Food color is in the eye of the beholder: the role of human trichromatic vision in food evaluation. Sci. Rep. 6, 37034. doi: 10.1038/srep37034

PubMed Abstract | CrossRef Full Text | Google Scholar

Graven, T., and Desebrock, C. (2018). Bouba or kiki with and without vision: Shape-audio regularities and mental images. Acta Psychol. 188, 200–212. doi: 10.1016/j.actpsy.2018.05.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Harrar, V., and Spence, C. (2013). The taste of cutlery: how the taste of food is affected by the weight, size, shape, and colour of the cutlery used to eat it. Flavour 2, 21. doi: 10.1186/2044-7248-2-21

CrossRef Full Text | Google Scholar

Huang, F., Huang, J., and Wan, X. (2019). Influence of virtual color on taste: Multisensory integration between virtual and real worlds. Comput. Hum. Behav. 95, 168–174. doi: 10.1016/j.chb.2019.01.027

CrossRef Full Text | Google Scholar

Huisman, G., Bruijnes, M., and Heylen, D. K. J. (2016). “Moving feast: effects of color, shape and animation on taste associations and taste perceptions,” in ACM International Conference Proceeding Series, 1–12. doi: 10.1145/3001773.3001776

CrossRef Full Text | Google Scholar

Istiani, N. F. F., Maffei, L., and Masullo, M. (2022). “The influence of multisensory design in indoor environment on the fruit juice drinking experience,” in Proceedings of the 24th International Congress on Acoustics October 24–28th 105–116.

Google Scholar

Koch, C., and Koch, E. C. (2003). Preconceptions of taste based on color. J. Psychol. Interdiscip. Appl. 137, 233–242. doi: 10.1080/00223980309600611

PubMed Abstract | CrossRef Full Text | Google Scholar

Köhler, W. (1929). Gestalt Psychology. New York: Liveright.

Google Scholar

Kurihara, K. (2015). Umami the fifth basic taste: history of studies on receptor mechanisms and role as a food flavor. Biomed Res. Int. 2015, 189402. doi: 10.1155/2015/189402

PubMed Abstract | CrossRef Full Text | Google Scholar

Lenth, R., Singmann, H., Love, J., Buerkner, P., and Herve, M. (2020). Package “emmeans.” CRAN Repos. 34, 216–221.

Google Scholar

Levitan, C. A., Ren, J., Woods, A. T., Boesveldt, S., Chan, J. S., McKenzie, K. J., et al. (2014). Cross-cultural color-odor associations. PLoS ONE 9, e101651. doi: 10.1371/journal.pone.0101651

PubMed Abstract | CrossRef Full Text | Google Scholar

Levitan, C. A., and Shermer, D. Z. (2014). Red hot: the crossmodal effect of color intensity on perceived piquancy. Multisens. Res. 27, 207–223. doi: 10.1163/22134808-00002457

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, R., Hannum, M., and Simons, C. T. (2019). Using immersive technologies to explore the effects of congruent and incongruent contextual cues on context recall, product evaluation time, and preference and liking during consumer hedonic testing. Food Res. Int. 117, 19–29. doi: 10.1016/j.foodres.2018.04.024

PubMed Abstract | CrossRef Full Text | Google Scholar

Motoki, K., Takahashi, A., and Spence, C. (2021). Tasting atmospherics: Taste associations with colour parameters of coffee shop interiors. Food Qual. Prefer. 94, 104315. doi: 10.1016/j.foodqual.2021.104315

CrossRef Full Text | Google Scholar

Narumi, T., Nishizaka, S., Kajinami, T., Tanikawa, T., and Hirose, M. (2011). “Augmented reality flavors: Gustatory display based on Edible Marker and cross-modal interaction,” in Conference on Human Factors in Computing Systems - Proceedings 93–102. doi: 10.1145/1978942.1978957

CrossRef Full Text | Google Scholar

Ngo, M. K., Velasco, C., Salgado, A., Boehm, E., O'Neill, D., and Spence, C. (2013). Assessing crossmodal correspondences in exotic fruit juices: the case of shape and sound symbolism. Food Qual. Prefer. 28, 361–369. doi: 10.1016/j.foodqual.2012.10.004

CrossRef Full Text | Google Scholar

Nishizawa, M., Jiang, W., and Okajima, K. (2016). “Projective-AR system for customizing the appearance and taste of food,” in Proceedings of the 2016 Workshop on Multimodal Virtual and Augmented Reality, MVAR 2016 - In Conjunction with ACM ICMI 2016, 1–6. doi: 10.1145/3001959.3001966

CrossRef Full Text | Google Scholar

North, A. C. (2012). The effect of background music on the taste of wine. Br. J. Psychol. 103, 293–301. doi: 10.1111/j.2044-8295.2011.02072.x

CrossRef Full Text | Google Scholar

Nygård, M., and Lie, C. L. (2020). How changes in environmental colour hue affect taste expectations, perceptions, and product preferences at different levels of attention towards atmospheric cues: A mixed experimental design. Master's thesis, Handelshøyskolen BI.

Google Scholar

Obrist, M., Comber, R., Subramanian, S., Piqueras-Fiszman, B., Velasco, C., and Spence, C. (2014). “Temporal, affective, and embodied characteristics of taste experiences: A framework for design,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2853–2862. doi: 10.1145/2556288.2557007

CrossRef Full Text | Google Scholar

O'Mahony, M. (1983). Gustatory responses to nongustatory stimuli. Perception 12, 627–633. doi: 10.1068/p120627

PubMed Abstract | CrossRef Full Text | Google Scholar

Pereira-Castro, M. R., Pinto, A. G., Caixeta, T. R., Monteiro, R. A., Bermúdez, X. P. D., and Mendonça, A. V. M. (2022). Digital forms of commensality in the 21st Century: a scoping review. Int. J. Environ. Res. Public Health 19, 16734. doi: 10.3390/ijerph192416734

CrossRef Full Text | Google Scholar

Picket, B., and Dando, R. (2019). Environmental immersion's influence on hedonics, perceived appropriateness, and willingness to pay in alcoholic beverages. Foods 8, foods8020042. doi: 10.3390/foods8020042

PubMed Abstract | CrossRef Full Text | Google Scholar

Pinheiro, J. C., and Bates, D. M. (2000). “Linear mixed-effects models: basic concepts and examples,” in Mixed-Effects Models in S and S-Plus. doi: 10.1007/978-1-4419-0318-1

CrossRef Full Text | Google Scholar

Piqueras-Fiszman, B., and Spence, C. (2011). Do the material properties of cutlery affect the perception of the food you eat? An exploratory study. J. Sensory Stud. 26, 358–362. doi: 10.1111/j.1745-459X.2011.00351.x

CrossRef Full Text | Google Scholar

Piqueras-Fiszman, B., and Spence, C. (2015). Sensory expectations based on product-extrinsic food cues: an interdisciplinary review of the empirical evidence and theoretical accounts. Food Qual. Prefer. 40, 165–179. doi: 10.1016/j.foodqual.2014.09.013

CrossRef Full Text | Google Scholar

R Development Core Team (2019). R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing.

Google Scholar

Riofrio-Grijalva, R., Lago, M., Fabregat-Amich, P., Guerrero, J., Cuesta, A., and Vázquez-Araújo, L. (2020). Relationship between tactile stimuli and basic tastes: CATA with consumers with visual disability. J. Sens. Stud. 35, e12549. doi: 10.1111/joss.12549

CrossRef Full Text | Google Scholar

Salgado-Montejo, A., Alvarado, J. A., Velasco, C., Salgado, C. J., Hasse, K., and Spence, C. (2015). The sweetest thing: the influence of angularity, symmetry, and the number of elements on shape-valence and shape-taste matches. Front. Psychol. 6, 1382. doi: 10.3389/fpsyg.2015.01382

PubMed Abstract | CrossRef Full Text | Google Scholar

Saluja, S., and Stevenson, R. J. (2018). Cross-modal associations between real tastes and colors. Chem. Senses 43, 475–480. doi: 10.1093/chemse/bjy033

PubMed Abstract | CrossRef Full Text | Google Scholar

Shankar, M., Simons, C., Shiv, B., Mcclure, S., Levitan, C., and Spence, C. (2010). An expectations-based approach to explaining the cross43 modal influence of color on orthonasal olfactory identification: The influence of the degree of discrepancy. Atten Percept Psychophys. 72, 1981–1993. doi: 10.3758/APP.72.7.1981

PubMed Abstract | CrossRef Full Text | Google Scholar

Slocombe, B. G., Carmichael, D. A., and Simner, J. (2016). Cross-modal tactile–taste interactions in food evaluations. Neuropsychologia 88, 58–64. doi: 10.1016/j.neuropsychologia.2015.07.011

CrossRef Full Text | Google Scholar

Spence, C. (2011). Crossmodal correspondences: a tutorial review. Attent. Percept. Psychophys. 73, 971–995. doi: 10.3758/s13414-010-0073-7

CrossRef Full Text | Google Scholar

Spence, C. (2015). On the psychological impact of food colour. Flavour 4, 1–16. doi: 10.1186/s13411-015-0031-3

CrossRef Full Text | Google Scholar

Spence, C. (2020). Temperature-based crossmodal correspondences: causes and consequences. Multisens. Res. 33, 645–682. doi: 10.1163/22134808-20191494

PubMed Abstract | CrossRef Full Text | Google Scholar

Spence, C. (2023). Digitally enhancing tasting experiences. Int. J. Gastron. Food Sci. 32, 100695. doi: 10.1016/j.ijgfs.2023.100695

CrossRef Full Text | Google Scholar

Spence, C., and Levitan, C. A. (2021). Explaining crossmodal correspondences between colours and tastes. Iperception. 12, 20416695211018223. doi: 10.1177/20416695211018223

PubMed Abstract | CrossRef Full Text | Google Scholar

Spence, C., Velasco, C., and Knoeferle, K. (2014). A large sample study on the influence of the multisensory environment on the wine drinking experience. Flavour 3, 1–12. doi: 10.1186/2044-7248-3-8

CrossRef Full Text | Google Scholar

Spence, C., and Youssef, J. (2019). Synaesthesia: the multisensory dining experience. Int. J. Gastron. Food Sci. 18, 100179. doi: 10.1016/j.ijgfs.2019.100179

CrossRef Full Text | Google Scholar

Strugnell, C. (1997). Colour and its role in sweetness perception. Appetite 28, 85. doi: 10.1006/appe.1996.0065

PubMed Abstract | CrossRef Full Text | Google Scholar

Tarrega, A., Marcano, J., and Fiszman, S. (2016). Yogurt viscosity and fruit pieces affect satiating capacity expectations. Food Res. Int. 89, 574–581. doi: 10.1016/j.foodres.2016.09.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Tomasik-KrÓtki, J., and Strojny, J. (2008). Scaling of sensory impressions. J. Sens. Stud. 23, 251–266. doi: 10.1111/j.1745-459X.2008.00153.x

CrossRef Full Text | Google Scholar

Ueda, J., Spence, C., and Okajima, K. (2020). Effects of varying the standard deviation of the luminance on the appearance of food, flavour expectations, and taste/flavour perception. Sci. Rep. 10, 16175. doi: 10.1038/s41598-020-73189-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Velasco, C., Jones, R., King, S., and Spence, C. (2013). Assessing the influence of the multisensory environment on the whisky drinking experience. Flavour 2, 23. doi: 10.1186/2044-7248-2-23

CrossRef Full Text | Google Scholar

Velasco, C., and Obrist, M. (2021). Multisensory experiences: a primer. Front. Comput. Sci. 3, 614524. doi: 10.3389/fcomp.2021.614524

CrossRef Full Text | Google Scholar

Velasco, C., Woods, A. T., Deroy, O., and Spence, C. (2015). Hedonic mediation of the crossmodal correspondence between taste and shape. Food Qual. Prefer. 41, 151–158. doi: 10.1016/j.foodqual.2014.11.010

CrossRef Full Text | Google Scholar

Vickers, Z. M. (1983). Pleasantness of Food Sounds. J. Food Sci. 48, 783–786. doi: 10.1111/j.1365-2621.1983.tb14898.x

CrossRef Full Text | Google Scholar

Wan, X., Woods, A. T., Van Den Bosch, J. J. F., McKenzie, K. J., Velasco, C., and Spence, C. (2014). Cross-cultural differences in crossmodal correspondences between basic tastes and visual features. Front. Psychol. 5, 1365. doi: 10.3389/fpsyg.2014.01365

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, Q., and Spence, C. (2015). Assessing the influence of the multisensory atmosphere on the taste of vodka. Beverages 1, 204–217. doi: 10.3390/beverages1030204

CrossRef Full Text | Google Scholar

Wang, Q. J., Keller, S., and Spence, C. (2017). Sounds spicy: Enhancing the evaluation of piquancy by means of a customised crossmodally congruent soundtrack. Food Qual Prefer. 58, 1–9. doi: 10.1016/j.foodqual.2016.12.014

CrossRef Full Text | Google Scholar

Weidner, F., Maier, J. E., and Broll, W. (2023). Eating, smelling, and seeing: investigating multisensory integration and (in)congruent stimuli while eating in VR. IEEE Trans. Vis. Comput. Graph. 29, 2423–2433. doi: 10.1109/TVCG.2023.3247099

PubMed Abstract | CrossRef Full Text | Google Scholar

Wickham, H. (2016). ggplot2: Elegant Graphics for Data Analysis. Available online at: https://ggplot2.tidyverse.org (accessed June 28, 2023).

Google Scholar

Witmer, B. G., and Singer, M. J. (1998). Measuring presence in virtual environments: a presence questionnaire. Pres. Teleoper. Virtual Environ. 7, 225–240. doi: 10.1162/105474698565686

CrossRef Full Text | Google Scholar

Wu, Z., Shi, R., Li, Z., Jiang, M., Li, Y., Yu, L., et al. (2022). Examining cross-modal correspondence between ambient color and taste perception in virtual reality. Front. Virtual Real. 3, 1056782. doi: 10.3389/frvir.2022.1056782

CrossRef Full Text | Google Scholar