Examining cross-modal correspondence between ambient color and taste perception in virtual reality

This research explores the cross-modal correspondence effect of ambient color on people’s taste perception in virtual reality (VR). To this end, we designed and conducted two experiments to investigate whether and how taste-congruent ambient colors in VR influence taste perception measured by four aspects: 1) taste ratings of a neutral drink; 2) taste association with virtual environments; 3) associated scenarios when immersed in these virtual environments; and 4) participants’ liking of these environments. In Experiment 1, participants adjusted the ambient light with different cross-modal-related colors in the immersive environments and reported their scaling of the Virtual Reality Sickness Questionnaire (VRSQ). Comfortable light intensity for each ambient color was obtained and color recognition problems were observed. In Experiment 2, participants tasted black tea (as the neutral drink), after being exposed to eight different virtual environments with different ambient colors. Results showed that the pink ambient color significantly increased the sweetness ratings. Differences in the color-taste association and environment liking were also observed in the ambient color conditions. Our results provide new insights into the cross-modal correspondence effect on ambient color and taste perception not found in prior work in VR scenarios.

1 Introduction

Cross-modal correspondence (CMC) is the interaction among various senses, where people map sensations in one sensory modality onto other modalities in a consistent manner (Spence, 2011). Taste is one of the most common sensations that people experience daily. Basic taste perceptions include sweetness, sourness, bitterness, saltiness, and umami¹ (Keast and Breslin, 2003). Prior studies have explored how taste perception can be formed with other senses via CMC (Knöferle and Spence, 2012; Kerruish, 2019; Halabi and Saleh, 2021). Existing research has provided evidence that color is an intrinsic and extrinsic visual stimulus influencing taste. Intrinsic colors are those features of the food itself, including the color of the food (Maga, 1974; Johnson and Clydesdale, 1982; Clydesdale. et al., 1992; Charles et al., 2010; Spence and Piqueras-Fiszman, 2016). Extrinsic colors are those external to the food, including the influence of color itself (Koch and Koch, 2003; Wan et al., 2014), the color of the container (like the cup or the bowl) (Harrar et al., 2011; Piqueras-Fiszman and Spence, 2012; Spence and Velasco, 2018; Sugimori and Kawasaki, 2022), and the color of the environment (Oberfeld et al., 2009).

Recently, there has been an increasing trend of using relatively newer interactive technologies to enhance or modify taste perception (Gallace et al., 2012; Moser and Tscheligi, 2013; Nishizawa et al., 2016; Arnold, 2017; Kerruish, 2019; Halabi and Saleh, 2021). For example, Narumi et al. (2011) proposed MetaCookie+, a pseudo-gustatory system that uses an augmented reality head-mounted display (AR HMD) to overlay both visual and olfactory information on real cookies to observe taste alterations. Another work by Nishizawa et al. (2016) introduced a projective-AR system to modify the flavor and taste of chips by changing their hue properties. Besides AR environments, researchers found immersive virtual reality (VR) environments can also influence people’s food or beverage taste experience. VR is regarded as a new medium to provide visual stimulation and is gradually becoming a research tool for the discipline of perception (Kuliga et al., 2015; Lipson-Smith et al., 2020). Previous works show that this medium has several benefits. The users can have higher engagement and taste perception consistency in an immersive virtual environment compared to a physical environment (Carlson et al., 2011; Ledoux et al., 2013; Bangcuyo et al., 2015; Stelick et al., 2018). VR also presents a cost-effective way to provide various stimuli (Lipson-Smith et al., 2020) and thus can be used to conduct empirical studies on psychology or perception.

Researchers have utilized this medium to study the effects of color on taste perception, especially the beverage’s (intrinsic) color on its tasting. For example, research from Wang et al. (2020) modified the color of the coffee in VR and Huang et al. (2019) studied the influence of tea color simulation in VR. In addition to changing the color of the food, Chen et al. (2020) studied the synthesis of multiple visual elements, with the environmental color as one of the variables. Although these works explored the feasibility of conducting taste perception studies related to color in virtual environments, they either focused on manipulating the intrinsic color of the foodstuff or dealing with complex VR scenes with various visual elements or items than colors only. Compared with the modification of the intrinsic colors or textures, the focus on environmental color may be more suitable as VR environments emphasize the visual immersion of users. However, from Chen et al. (2020)’s experiment, we can not decide whether the change in taste perception was because of the color or other visual stimuli. A simple and specific type of extrinsic visual stimulus from the virtual environment is worthy of investigation because we can then isolate the effect of specific visual aspects of such environments and leverage the findings to develop VR environments tailored to more specific purposes.

In this work, we investigate the influence of ambient light color variations in virtual environments. This type of stimulus has been studied in physical settings, presenting a significant influence on taste perception (Oberfeld et al., 2009; Spence et al., 2014), but it has not been explored in VR environments. Ambient light color is ubiquitous in such environments and fits the characteristic of immersion as an extrinsic indirect visual cue. However, limited research has investigated how could this parameter in immersive virtual environments influences taste perception. To fill this research gap in our understanding of how ambient light affects taste perceptions in virtual environments, we run two experiments that help answer two research questions (RQs).

RQ1: What will the suitable environment parameter be when we apply the cross-modal-related colors to the VR environments? is the main research question explored in Experiment 1. Previous work revealed two potential problems that color studies in VR environments may suffer: 1) participants may experience visually induced motion sickness (VIMS) due to the unpleasant colors (Bonato et al., 2004; So and Yuen, 2007; Gusev et al., 2018; Grassini et al., 2021); and 2) there may be a recognition deviation between the color of the ambient light presented through the VR HMD and the color perceived in the physical setting. Based on these, we conducted Experiment 1 to evaluate the impact of these issues and set up the ambient light conditions to be suitable for color-taste studies. We formulated the testing virtual environments with 11 ambient colors, including black, green, grey, orange, pink, purple, red, yellow, white, blue, and brown. Participants adjusted the light intensity of these environments to make them comfortable, then reported their perceived VIMS and recognized color when exposed to the environment. Our findings show that brown, white, and grey ambient lights were considered to be more comfortable. However, these three colors were not well recognized and thus were excluded in Experiment 2.

RQ2: How will taste perception be influenced by different ambient colors? is the research question for Experiment 2, where we further studied the influence of the ambient color setups via Experiment 1 on perceived taste in VR environments. For this research question, we investigated the following three aspects of taste perception: 1) participants’ perceived taste of the tastant in terms of sweetness, sourness, bitterness, and saltiness; 2) participants’ taste association under the stimuli together with the associated scenarios that participants imagined; and 3) participants’ preference of the tested virtual environments where they tasted the tastant. According to Wan et al. (2014), we excluded umami in our study because it did not show a strong association with any color, shape, or texture in real-life settings. Moreover, the term umami may be difficult to understand due to its inconsistent perceptions across different cultural backgrounds (Gotow et al., 2021). We selected black tea as our neutral tastant based on a pilot study. Our findings reveal that ambient color significantly influenced the sweet taste of the tastant. Pink and purple showed a significant association with the sweet taste rather than the bitter taste while black and green were associated with bitterness more frequently compared with sweetness. In terms of the associated scenarios coupled with the color-taste association, most of the participants associated the sweet taste with food-related items, but the bitter taste with specific scenes. Finally, orange ambient light was preferred over red.

Our work presents two main contributions:

• An experiment protocol that uses ambient color for taste perception studies in virtual environments.

• Evidence that ambient color is a significant factor for perceived sweetness and bitterness sensations in VR.

2 Related work

2.1 Taste perception in virtual environments

Taste experience can be introduced to different types of virtual environments, ranging from curved computer-based screens to AR/VR HMDs (Wang et al., 2021). There is an increasing trend of using virtual environments in human-food interaction research because they allow researchers to easily add environmental features or properties and minimize certain negative effects of real settings (Bangcuyo et al., 2015; Delarue et al., 2019). For instance, compared to traditional booths, the hedonic data collected in a virtual coffee house can be more identifiable and reliable as a predictor for future coffee evaluation (Bangcuyo et al., 2015). Delarue et al. (2019) have shown that a multi-sensory immersive room could minimize the differences in product testing results produced at different periods in a lab setting. Moreover, introducing the taste experience to VR also benefits the immersive experience. Eating and drinking unify both virtual and physical sensory cues (Spence, 2016). Thus, they can increase users’ presence in the virtual environments, i.e., their sense of being there (Slater and Wilbur, 1997). Wang et al. (2021) also suggested that introducing physical interaction mechanisms that the users are used to in the physical world into the immersive virtual environments can enhance the psychological feeling of presence. Furthermore, Gallace et al. (2012) stated that increasing the number of senses stimulated in VR can increase users’ enjoyment, memorability, and presence. Since taste is one of the major sensory modalities familiar to people in real-life settings, it is beneficial to have it in VR. Eating and food-orientating activities may also become integral to the virtual experience in social contexts as VR becomes more available and allow people to meet on platforms like VRChat² for socialization and Horizon Workrooms³ for business meetings.

On the other hand, taste-related results in virtual environments may be different from those obtained in physical environments. For example, the virtual environment may significantly influence users’ hedonic responses to foodstuffs. Beef can be rated higher in terms of liking when consumed in a VR restaurant compared to a traditional sensory laboratory condition (Crofton et al., 2021). Stelick et al. (2018) also found that VR had a significant influence on the taste perception of blue cheese, especially the level of pungency. Ledoux et al. (2013) showed that food craving produced by VR was marginally greater than a neutral cue in a physical setting.

2.2 Cross-modal effect between color and taste

Cross-modal correspondence (CMC) is about the interactions among different senses. People combine information captured from various senses to form a more comprehensive view of the external world or activity they are doing (e.g., when people see and smell food in front of them). Interaction between these systems is an essential part of this process (Stein and Meredith, 1993). In particular, based on CMC, people can map features or sensations in one sensory modality onto features of other modalities in a consistent manner (Spence, 2011). For example, the taste of sweetness can correspond to a round shape (visual) or high pitch (audio). Surprisingly, CMC is not confined to one group (cultural or otherwise) but is often universally shared (Knöferle and Spence, 2012). Taste is one of the most common sensations that people experience daily. Basic taste perceptions include sweetness, sourness, bitterness, saltiness, and savor (umami) (Keast and Breslin, 2003), which can often be formed with other senses, such as smell or sound (Knöferle and Spence, 2012; Kerruish, 2019; Halabi and Saleh, 2021).

Color is an essential aspect of CMC between vision and taste. Color-taste correspondence is of great interest to the food industry, especially in helping design food product packaging because of its direct influence on consumers’ expectations (Cardello, 1994; Koch and Koch, 2003). To a large extent, it serves as a sensory property of the product before its consumption (Spence and Velasco, 2018). Color is a focused factor that significantly affects taste perception (Piqueras-Fiszman and Spence, 2014). For example, from the studies of color itself, Koch and Koch (2003) have stated that red is connected to sweetness while yellow is connected to sour, citrus, and fruity tastes. In addition, Wan et al. (2014) have shown that some strong CMCs are similar in different cultures, such as mapping black with bitterness and pink with sweetness.

Factors influencing multi-sensory flavor perception can be divided into either intrinsic to the food (e.g., texture, aroma) or extrinsic to it (e.g., related to the packaging, receptacle, or external environment) (Wang et al., 2019), both of which have been studied by previous research on color-taste CMC. When color represents the original feature of the foodstuff, it is considered an intrinsic visual stimulus feature (Spence, 2019b). For example, Johnson and Clydesdale (1982) found that applying red color to water could increase the sweet taste. Similarly, Maga (1974) showed that green statistically increased sweet taste threshold sensitivity while yellow decreased taste sensitivity. Different from intrinsic features, extrinsic factors are outside of the food itself. For example, Piqueras-Fiszman and Spence (2012) have shown that orange and dark-cream cups could enhance the chocolate flavor of a drink. Harrar et al. (2011) investigated popcorn in four different colored bowls and found that the sweet popcorn, in addition to being sweet, was perceived as saltier when consumed in a colored bowl than in a white bowl.

Specifically, we noticed that the color of ambient light is a type of extrinsic factor that can influence taste. Spence and Carvalho (2020) stressed the importance of the environment where the beverages are consumed. The ambient color of an environment could significantly affect the beverage’s flavor and taste (Oberfeld et al., 2009). Oberfeld et al. (2009) showed that wine was perceived to be spicier in blue or green ambient lighting than in red or white. Spence et al. (2014) explored people’s taste perception of red wine in a room that changes ambient color and music and found a significant difference in taste and beverage liking. Their results provide empirical support for the claim about the influence of ambient color on flavor from Oberfeld et al. (2009).

In general, there are three possible mechanisms underpinning the CMCs (Spence, 2011; Parise and Spence, 2013): 1) structural correspondence, described from the neural science side, is the byproduct of the innate cognitive system; 2) statistical correspondence, describes a learned process associating the established experience and the environment; and 3) semantic correspondence, stresses that the same terms can be used to describe different stimuli linguistically. For color-taste CMC, there are mainly two types of specific explanations for their correspondence. The first type is the association with the source object(s), meaning that the color-taste pairs result from associating specific colors with certain gustatory cues (Wang et al., 2020). Moreover, Spence and Levitan (2021) found that some color-taste associations are not limited to a particular source object. The second explanation is users’ emotional mediation. For example, the pink-sweet association arises because both stimuli are linked with happiness independently and are connected by people (Spence and Levitan, 2021). It is important to note that some colors with positive emotional valance in abstract scenarios may lead to negative emotions when matching with food. For instance, blue is a favorite color to many people, but blue-colored food can trigger negative feelings (Spence, 2019a).

2.3 Color-taste study in VR

The color-related research in the physical world has inspired some researchers to apply them in VR. Wang et al. (2020) designed an experiment manipulating the color of coffee in VR so that their participants saw either a dark brown or light brown liquid as they consumed the real coffee. Similarly, Huang et al. (2019) designed three experiments to study the influence of actual tea color simulation in VR on users’ taste ratings of Chinese red and green tea. Moreover, Chen et al. (2020)’s study went beyond colors. They created a sweet, bitter, and neutral scene integrating shape, color, and visual textures, and assessed their influence on perceived sweetness and environment liking. Results showed that the sweet-congruent environment significantly increased the perceived sweetness of the beverage they tested. However, these studies either focus on manipulating the intrinsic color of the foodstuff in VR or dealing with complex VR scenes.

As discussed in the previous section, the color of the ambient light is an effective stimulus studied in the physical environment. This inspired us to consider if similar effects can be reproduced and leveraged in VR environments. The virtual environments may cost-effectively control or simulate the ambient color rooms while providing fully colored immersive environments. Firstly, the color of the ambient light is a simple parameter that can be controlled in VR and be compared with previous works done in the physical setting. Although not related to taste, researchers have already suggested using VR for ambient color studies. For example, Oberfeld and Hecht (2011) studied the influence of environmental color on the perceived height and width of interior rooms virtually. von Castell et al. (2018) also studied the effects of the hue, saturation, and luminance of color on the perceived height of interior spaces via a VR HMD. Lipson-Smith et al. (2020) did a study focusing on the mood and preference towards the virtual environments varying in colors. In the meantime, applying ambient light as an environmental parameter to VR leverages the immersion provided by this medium well. In VR development tools, ambient light hue is an important parameter in creating the visual appearance of the scene and its objects. Changing ambient light hue can effectively alter the color scheme of all the 3D models (objects) in the VR scene (Unity, 2019). Based on these, we chose ambient light varying in colors as our stimuli to study its influence on taste perception in VR. While ambient light is essential to and a foundation building block to all VR environments, its effect on taste perception has not been explored in detail. Our research aims to fill this important but underexplored area.

2.4 Potential issues for VR color-taste studies

In a VR environment, color may induce Visually Induced Motion Sickness (VIMS) and make the application unusable (Gusev et al., 2016). Bonato et al. (2004) simulated the optokinetic drum (spinning sphere) with specific color patterns in a virtual environment. They suggested that the chromaticity might increase the inconsistency between visual and vestibular inputs. However, So and Yuen (2007) found the change in the colors would not influence VIMS significantly in VR. Besides, colored environments may lead to a higher VIMS compared to monochrome environments (Gusev et al., 2018). Based on the literature, it is not clear whether the ambient color in VR would cause VIMS and, if so, to what extent the VIMS would be. To avoid discomfort and bias in our color-taste experiment, we conducted the first experiment to adjust the ambient colors in the virtual environments to ensure they were not harmful to the experience. Color recognition can be another potential issue for VR color-taste studies. It is important to consider whether people could depict the ambient color hue in a VR environment as they perceive it in the real world (Lipson-Smith et al., 2020). We also want to filter out this issue before running the color-taste experiment.

Selecting a suitable tastant is also important for a color-taste study. Some of the existing studies in CMC between taste and other stimuli used word-based descriptions or images of the food. Crisinel and Spence (2009) used Implicit Association Test and applied food images as visual stimuli to investigate the association between basic taste and pitch (auditory parameter). Similarly, Wan et al. (2014) used graphics with words referring to taste as the visual stimuli in their online study. The participants dragged the image to the word they thought it was associated with. On the contrary, Crisinel and Spence (2010) did a repeated experiment using real tastants. However, they got similar taste perception results as those evoked by the food names. As reported by Crisinel and Spence (2009) in their work, although it was convenient to use the names of food or drinks in the study, it entailed several disadvantages. First, between-participant differences might exist, such as the same word “beer” may associate with various distinctive tastes. Second, excluding real tastants prevented them from studying the direct interaction between the sensory modalities. As such, in our study, we did not use words and instead used a real tastant because this is more aligned with the use of VR, as a platform that brings users to experience virtual environments but allows them to still be closely associated with the physical world.

Previous studies in taste and virtual environments have utilized a variety of tastant choices. We collected some representative ones and listed also in Table 1. Among these, we found that a beverage is the most frequently used tastant. In addition, drinking is relatively more convenient and safer than eating. Thus, we used beverages as our taste stimuli. We anticipated that a beverage with a very strong taste would lead towards one specific taste and bias the results. Due to this, a neutral tastant that would not affect participants’ taste perception is required. However, limited research has discussed the rationale behind choosing the tastants and whether their tastes are neutral. We decided to run a pilot study to choose a neutral tastant before our formal color-taste experiment.

TABLE 1

TABLE 1. A summary of related work about taste and the immersive environments with the used stimuli. Tastants are highlighted in bold and tested stimuli were marked in italic.

2.5 Summary

Table 1 summarizes the related works mentioned above. This study has three main differences compared to prior studies:

1) Unlike prior work about taste perception in immersive virtual scenes (Section 2.1), our focus was CMC-related stimuli instead of realistic scene simulation.

2) In terms of research about visual-taste CMC in the physical environment (Section 2.2), our work is different from the experiments from Koch and Koch (2003) and Wan et al. (2014) because our work included a real tastant. In addition, our work differs from studies dealing with ambient light (e.g. Oberfeld et al., 2009; Spence et al., 2014). We introduce VR as a platform to determine its level of visual-taste CMC. To do this, several changes and considerations are required but not found in prior research. The use of VR as a medium may bring interdisciplinary insights to applications in VR, which has grown rapidly in the last few years and is positioned to shift significantly how we interact with others, whether for social occasions (e.g., in platforms such as VRChat) or for production purposes (e.g., within Horizon Workrooms) that aim to emulate many aspects of the physical world.

3) Unlike previous work on taste-related CMC in VR (Section 2.3), our selection of ambient colors as the visual stimuli is novel and has not yet been explored systematically before. Ambient colors are foundational to VR applications due to their emphasis on visual immersive and focus on extrinsic modification. Our understanding of how ambient colors affect people’s perception of taste will enable us to develop VR applications that can be more tailored to specific situations.

3 Experiment 1: Adjustment of the ambient color in the virtual environments (RQ1)

In this experiment, we aim to explore and determine the suitable settings of different ambient color environments, which is the premise for their usage in the follow-up taste perception study. As mentioned in Section 2.4, color in VR environments may cause visually induced motion sickness (VIMS) (Bonato et al., 2004; So and Yuen, 2007; Gusev et al., 2018; Grassini et al., 2021). Directly adapting the colors from former color-taste studies in physical settings to the ambient color in virtual environments with the default light setting may cause discomfort to users. Adjustments to the related lighting parameters for these colors with high visual saturation are necessary. In addition to the hue (i.e., RGB values), there are several parameters influencing the visual presentation and rendering of the environment: ambient mode, compression, environment reflections, light intensity, and bounces in Unity engine (Unity, 2019). We set our main adjustment parameter as light intensity since it directly influences the brightness of the ambient light in the VR scene. Also, as mentioned in Section 2.4, it is uncertain whether color recognition in VR deviates from the physical world, which might lead to ambiguous or biased results in color-taste studies. Thus, we also tested the recognition accuracy of the ambient color in VR in Experiment 1.

3.1 Stimuli

We utilized the colors described in Wan et al. (2014)’s taste-color synesthesia, including black, green, grey, orange, pink, purple, red, yellow, white, blue, and brown. These colors were used as the hue of the ambient light in the VR environments. The RGB values of these colors are also listed in Figure 1. Specifically, we created 11 hexagonal virtual rooms to demonstrate these ambient colors (see Figure 1). In each room, we placed a spotlight at the top middle of the room to make sure it illuminated the room (i.e., the ambient color covered the entire interior of the room), and assigned the RGB value of the color. The spot angle of the light was 129.8°. We set the range for controlling how far the light is emitted from the center of the object from 0 to 20. The default light intensity for each room is 10. We left the rooms empty to prevent any confounding factors.

FIGURE 1

FIGURE 1. Ambient color stimuli that were utilized in this study, with their RGB values, ambient light effects in VR, and the suitable light intensities obtained from Experiment 1. The first row with an arrow represents the colors tested in Experiment 1. The second row indicates the colors in Experiment 2, with white as a baseline color.

3.2 Participants and apparatus

Twelve healthy participants (6 females, 6 males; mean age = 21.2 ± 1.25 years) from a Sino-Chinese university located in a mid-size city in China were recruited for the first experiment. The participants were not paid and joined the experiment voluntarily. They did not have color blindness and did not report any issues regarding color recognition. Seven participants had prior experiences with VR HMD.

We used an Oculus Rift S VR HMD to display the virtual environments. The HMD has a resolution of 1,280 × 1,440 px per eye, a refreshing rate of 80 Hz, and a 115° field of view. Participants could move their heads in 6DOF. It was connected to an Intel Core i7-6700HQ processor laptop with an NVIDIA GeForce GTX 1070 GPU card. Participants could interact with the virtual environment via an Oculus hand-held controller. The virtual environments were created using the Unity3D platform (version 2019.1.14f1) and the related scripts were programmed in C#. The experiment was conducted in an empty and quiet room where the participants were equipped with the VR HMD and controllers. Figure 2 shows the setup.

FIGURE 2

FIGURE 2. Experiment 1 setup. A participant is adjusting the light intensity of pink ambient color in the virtual room displayed via an VR HMD, which is connected to a laptop.

3.3 Experimental design and task

A within-subjects design was used in this experiment. Participants would enter all the virtual rooms to complete the given tasks. When first entering a room, the initial intensity of the tested ambient light was set at 10. Participants were required to adjust the light intensity up and down starting from this value until they found the lighting condition that was the most comfortable. For experimental purposes, we set the adjustment range of the light intensity from 0 (dimmest light) to 20 (brightest light for our 3D assets). Participants could press Button A or B to make small adjustments to the light intensity (increased or decreased by 0.5 per press) and push the thumbstick to make big adjustments (see Figure 2). Once they settled down the light intensity for the current room, they stopped interacting with the controller for about 10 s. The finalized light intensity was then recorded by the program. After that, they needed to answer the question “what ambient color do you think is the current environment?” and complete a Virtual Reality Sickness Questionnaire (VRSQ) (Kim et al., 2018) in oral form. The experimenter asked these questions from VRSQ one by one and manually recorded the answers. Then the participants moved to the next room. The order of the room for each participant was counterbalanced using a Latin-Square design. Note that in our experimental design, participants’ answers toward color recognition might not have a relationship with their adjustment of light intensity. Although each participant had encountered several levels of intensity, they were exposed to the same intensity condition at first and were informed that they would make the adjustment for one specific color each time.

3.4 Procedure

The whole experiment lasted approximately 30 min for each participant. Before the formal experiment began, participants were asked to fill in a questionnaire for demographic background. We asked about their prior VR experience and if they had difficulty recognizing colors. Then participants were introduced to the research aims, tasks, controls of the experiment, and the questions that would be asked. Then they wore the VR HMD and started the experiment. After they adjusted the light intensity for all the ambient colors, they would take off the HMD and have a break. After the break, we asked participants to adjust the light intensity for the ambient colors in reverse order. This was intended to eliminate the residue effects from the previous colored room.

3.5 Results and discussion

The average light intensities for each ambient color are 14.63 for black, 5.72 for green, 11.97 for grey, 6.82 for orange, 7.98 for pink, 6.63 for purple, 3.68 for red, 4.34 for yellow, 4.69 for white, 4.09 for blue, and 12.75 for brown. As mentioned before, the values are between 0 (the dimmest condition) to 20 (the brightest condition).

In total, we collected 24 color recognition results (12 participants × 2 rounds). Figure 3 summarizes the recognition accuracy for each ambient color. The recognition accuracy for the ambient color of green, blue, red, purple, pink, and orange was high. However, the ambient color of brown and grey was low (29% and 41.5%, respectively). Though not low in recognition accuracy, white was frequently mixed with light pink (4 times). For grey and brown, there was a grey-green, grey-blue confusion, and a brown-pink confusion. Figure 4 is the summary of the average VRSQ score for each ambient color following the calculating method provided by Kim et al. (2018). A higher score indicates that the simulator sickness caused by the ambient color environment is more severe. Participants regarded brown, grey, and white as more comfortable than other colors in the VR environment after self-adjusting the light intensity. However, the difference was not significant.

FIGURE 3

FIGURE 3. Recognition accuracy (in percentage) for each ambient color.

FIGURE 4

FIGURE 4. Average VRSQ scores for each ambient color, the error bars represent the standard deviations.

Colors with low recognition accuracy were excluded from the follow-up experiment. We excluded white as it was mixed with pink while keeping pink since pink could be recognized successfully in the virtual environment. Grey and brown were excluded for the same reason. Although the color recognition accuracy of black was not high, it is an important color showing a high correlation with the bitter taste in the physical environment. So, it was useful to keep it and see its effectiveness in the virtual environment.

The results of recognition accuracy show that gaps may exist between the ambient colors that a developer wants the audience to perceive and the actual perceived color by this audience, especially when creating virtual environments. Developers are suggested to pay attention to color recognition accuracy, especially when color is an important factor in the virtual environment. Another contributing result was the comfortable intensity for the 11 ambient colors in the virtual environment with their VRSQ value, working as a guideline for developers who want to utilize these colors in their designs.

4 Experiment 2: CMC between ambient color and taste in virtual environments (RQ2)

In this experiment, we investigated how ambient colors would influence users’ perception of taste in immersive virtual environments, including sweet, sour, bitter, and salty tastes, corresponding to RQ2. As mentioned in Section 2.4, we needed a real tastant for the color-taste experiment rather than using words describing different tastes. In addition, the tastant should be as neutral as possible in its perceived taste so that it would not affect participants’ taste perception. Thus, we run a pilot study to find a neutral tastant before the formal experiment.

4.1 Pilot study

We invited the same 12 participants in Experiment 1 to complete this pilot study. The participants filled in a demographic questionnaire before the formal experiment. In the questionnaire, we inquired about their attention to taste in the form of scaling questions from 0-low to 6-high, “to how much extent do you pay attention to taste during your daily dieting?”. Their average ratings on taste attention were: 5.08 for sweetness, 4.69 for sourness, 3.38 for bitterness, and 5.06 for saltiness. We prepared four common beverages, including apple juice (brand: Huiyuan Juice), black tea (brand: Nongfu spring), coconut water (brand: Goodfarmer), and yogurt soda (brand: Energetic Forest), all of which could be bought at a local grocery shop. Our main consideration was to provide several possible choices of drinks that do not have a strong taste and are also familiar to users. After some trials before the pilot study with a number of possibilities, the four chosen ones represent a wide range. The pilot study then helped us narrow done to one.

We opted to use mini paper cups (100 ml) to ensure that it was convenient to drink the beverage while wearing an HMD in the main experiment later. The cups were covered with lids to prevent participants from recognizing the beverage by seeing the content or by smelling the odor (see Figure 5). Participants were asked to drink each of the four beverages, holding it in their mouth for at least 3 s before swallowing it. Then they needed to rate the taste of the beverage in terms of sweetness, sourness, bitterness, saltiness, and neutrality using a 7-scale questionnaire. Water was given to participants between two beverages to remove any residual, left-over taste. The sequence of the delivered beverages was counterbalanced. Note that, water was not used as the tastant because it was used as the “palate cleanser” to clear participants’ taste from the previous drink, and as such using it as the tastant is not appropriate (Huang et al., 2019; Chen et al., 2020; Wang et al., 2020).

FIGURE 5

FIGURE 5. Tastant choices and the container of the tastant.

Black tea was rated the most neutral among the four beverages. The remaining tastants have strong sweet tastes based on the ratings. Therefore, we used black tea as the taste stimulus in our experiment.

4.2 Stimuli

We removed the colors showing low recognition accuracy from our Experiment 1 (as described in Section 3.5). Thus, the color stimuli in the main study included black, green, orange, pink, purple, red, yellow, and blue. Although white was excluded, we set it as a starting, ending, and transition color for the experiment to avoid the residue effects from the previous colored room. A similar approach can be found in Spence et al. (2014). Figure 1 shows a preview of each virtual environment with the RGB parameter and intensity level of each ambient color. Notice that we did not include the 3D model of the beverage in the virtual environment because we wanted to focus on the influence of the immersive environment only. We used black tea as the tastant based on the results of the pilot study. Like the pilot study, the tastant was served in 100 ml paper cups covered with lids.

4.3 Participants and apparatus

We recruited another 16 participants (7 females, 9 males; 22.2 ± 1.66 years, ranging from 21 to 27 years). Thirteen participants had used VR HMD before the experiment and all of them had normal or corrected-to-normal vision and had no history of color blindness. Regarding their drinking behaviors, thirteen participants reported drinking beverages several times a week and twelve participants pay much attention to the taste of the beverage during their daily drinking activities. We used the same VR apparatus as in Experiment 1.

4.4 Design and task

Figure 6 provides an overview of Experiment 2. We applied a within-subjects design in this experiment with ambient color as the independent variable. As mentioned, we used a white room as a starting, ending, and transition room between the two rooms. The order of ambient color conditions was counterbalanced with a Latin-Square design. In each condition, the participants would be exposed to the virtual room for 30 s, starting from when they entered the scene and ending when they heard a notification to drink the tastant. They were asked to have a sip from the 100 ml tastant in their mouths for 3 s before swallowing it. After they drank the tastant, they were then asked to report their feelings according to the following questions:

• Q2-1. Can you rate the taste of the drink in the just-finished scene in terms of sweet, sour, bitter, and salty? Ratings are from 1 (very weak) to 7 (very strong).

• Q2-2. What type of taste among sweetness, sourness, bitterness, and saltiness do you think matches the VR environment the most? You can also choose none if you cannot come up with an association.

• Q2-3. Do you have some scenarios in your mind when coming up with this association when answering the second question? You should first answer yes or no, and then provide a specific answer if you say yes.

• Q2-4. How much do you like the just-finished scene when drinking the beverage? Ratings are from 1 (dislike it very much) to 7 (like it very much).

FIGURE 6

FIGURE 6. Overview of experiment 2.

Before going to the next ambient color room, they were exposed to a white room and delivered a mini cup of pure water for palate cleansing. After the participants experienced all eight conditions, they were asked to take the VR HMD off, and answer a post-experiment questionnaire to collect their demographic information.

4.5 Procedure

The participants were first briefed about the task and procedure of the experiment. They were encouraged to raise any questions if something was unclear. They were also informed that they could stop at any point during the experiment if they felt uncomfortable. The participants then put the VR HMD on and were asked not to take it off before the end of the experiment. Thus, they could not see and did not receive any information about the drinks during the whole experiment. They were first exposed to a white room and were given a cup of pure water to accommodate the ‘blinded’ drinking action and cleansing palate. After that, they were given the mini cup containing the tastant and kept it in their hand. Next, the experimenter switched the scene to the first room to start the formal experiment. After the participants were immersed in the ambient color virtual environment for 30 s, the background voice began to remind the participant of starting drinking. After the drinking is finished, participants were asked for answering the above questions orally. Before going to the next ambient color room, they were exposed to a white room and the cups in their hands were replaced with new ones containing pure water. They then drank the pure water for palate cleansing. After that, they entered the next ambient color room with a new mini cup of tastant delivered, replacing the cup containing pure water. Noticing that, when immersed in the virtual environment without seeing the content in the physical world, most of the participants were unaware that they were drinking the same tastants throughout the study.

After the participants experienced all eight conditions, they were asked to take the VR HMD off, drink the taste stimuli once more, and then report their feelings. Finally, we gave participants a post-experiment questionnaire to collect their demographic information, past VR experience, and beverage consumption habits.

4.6 Results

SPSS (version 26) was used for data analysis. The results were transformed with Aligned Rank Transform (Wobbrock et al., 2011). We performed one-way Repeated Measure-(RM-) ANOVA with the transformed data for Q1-1 and Q1-4. If the transformed results violated the assumption of sphericity, we report the results with Greenhouse-Geisser or Huynh-Feldt corrections according to the case (ϵ < 0.75 or $>0.75$, respectively). We used Fisher’s exact test to examine whether there is a significant color-taste association (i.e., Q1-2). Post-hoc pairwise comparisons were run with Bonferroni corrections if the above tests revealed significant differences or associations.

4.6.1 Q2-1. Perceived taste of the drink

Q2-1 refers to the influence on real taste. Figure 7 summarizes the descriptive statistics and the distribution of the ratings of Q2-1. As can be seen from the figure, the VR room with pink ambient color (AC) was on average rated the highest for sweet taste (M = 2.500, SD = 1.119) among eight AC conditions. For sour taste, the VR room with green AC reached the highest average rating (M = 2.312, SD = 1.310). Similarly, bitter taste in this AC environment was also rated the highest (M = 3.250, SD = 1.436). Participants did not report a strong feeling of salty taste. The ratings on saltiness were all low, but the VR room with black AC reached the highest average score of salty (M = 1.688, SD = 1.158).

FIGURE 7

FIGURE 7. The descriptive statistics and the distribution of the results of taste rating (Q2-1). The dots, triangles, lines and diamonds represent participants’ ratings, average ratings, median and outliers, respectively.

RM-ANOVA tests revealed a significant main effect of AC on the ratings of sweet taste $(F(\mathrm{7,105})=2.296,p=0.032,{\eta }_p^2=0.133)$, but not on sour taste $(F(\mathrm{7,105})=1.547,p=0.159,{\eta }_p^2=0.094)$, bitter taste $(F(\mathrm{7,105})=1.106,p=0.365,{\eta }_p^2=0.069)$, or salty taste $(F(3.716,55.736)=0.957,p=0.466,{\eta }_p^2=0.060)$. We further performed a post-hoc analysis with the ratings of sweet taste. However, the pairwise comparison did not show significant differences in the ratings of sweet taste in the different AC conditions.

4.6.2 Q2-2. Color-taste associations

Q2-2 focuses on scenario-taste matching. Figure 8 shows the frequencies of taste matching for different ambient color environments. Results of Fisher’s exact test show that there was a significant association between color and taste (χ² = 87.230, p < 0.001). The frequencies and pairwise comparisons are summarized in Table 2. Black and green showed a significant association with bitter taste compared with sweet. Pink and purple presented a significant association with sweetness rather than bitterness. No significant difference was observed in the other colors. According to the frequencies, sour was associated more frequently with yellow, while salty was associated more frequently with blue.

FIGURE 8

FIGURE 8. The frequencies and pairwise comparisons of the answers to Q2-2 in Experiment 2. The colors in the figure correspond to the ambient colors in the virtual envrionmes.

TABLE 2

TABLE 2. The frequencies and pairwise comparisons of the answers to Q2-2 in Experiment 2. The values in bold indicate the most frequently selected taste. Each subscript letter denotes a subset of color categories whose proportions do not differ significantly from each other at the 0.05 level reading across a row.

4.6.3 Q2-3. Association of VR environment to real scenarios

Figure 9 shows the participants’ reported real scenarios associated with the ambient light virtual environments, after matching them with the taste. Their answers were listed on the right side, with lines in colors indicating the association was under which ambient color virtual environment. Overall, there were three types of answers: scenes, food, and feelings. There were both answers related to foodstuff and non-foodstuff. Most participants associated sweet taste with food-related items. For example, after being exposed to the pink ambient light in the virtual environment, some participants came up with “candy” in their minds. Other common answers were lemon-yellow ambient light, apple-green ambient light, and oranges-orange ambient light. On the contrary, participants came up with specific scenes other than food (e.g., sea and night) after matching the ambient light with bitterness. Regarding the success of coming up with scenarios in their minds, associated scenarios were frequently described after being matched with sweetness (43, 21, 29, and 5 for the number of answers in sweet, sour, bitter, and salty taste correspondingly). Few participants described scenarios after matching the ambient color environment with the salty taste.

FIGURE 9

FIGURE 9. General associated objects and scenes under different taste choice (Q2-3). Participants gave the answers after matching the ambient color with the taste. Answers of associated scenario were categorized into food and non-food and listed in the right side. Lines were colored for indicating what ambient colors the answers were based on. Notice that, not all the conditions have associated scenarios.

4.6.4 Q2-4. Degree of liking of ambient colors

Figure 10 provides an overview of the participants’ answers to Q1-4. It can be seen from the figure that participants preferred orange and pink environments. Black, green, and red ambient environments were disliked by some participants. There was a significant main effect on the ratings in their preferred VR room (F (7, 105) = 2.463, p = 0.022). Post-hoc pairwise comparisons showed that ratings of orange (M = 4.313, SD = 1.158) was significantly higher than red (M = 3.000, SD = 1.173; p = 0.024, adjusted).

FIGURE 10

FIGURE 10. The descriptive statistics and the distribution of the results of color environment liking (Q2-4). The dots, triangles, lines and diamonds represent participants’ ratings, average ratings, median and outliers, respectively. Asterisks mark the comparison that differed significantly (p < 0.05) in the post-hoc pairwise comparison.

5 Discussion

5.1 Comparison with related work

In this work, we focused on exploring how environmental colors would affect people’s taste perception in VR than verifying a psychological phenomenon in VR. We used VR as a platform for providing new perceptions and a higher level of immersion to study the cross-modal correspondence (CMC) experience.

As listed in the Related Work (Section 2), our research can be positioned differently from prior work in physical and VR settings. Our results in taste association present an interesting comparison with the results given by studies conducted using a desktop computer screen as a medium for showing colors (Wan et al., 2014). Pink was significantly associated with sweetness in both VR and the real world. However, different from Wan et al. (2014)’s results from experiments in the real world where green was found to be more associated with sourness and black with a bitter taste, our results in VR showed that green can also be associated with a bitter taste. This indicates that the same color studied via different mediums may bring the same or different results in taste perception.

Unlike prior work regarding taste-related CMC in virtual environments, our selection of ambient colors as visual stimuli is novel and has not been explored systematically before. Choosing a parameter that is not related to the environment (e.g., color change of the foodstuff, change of the container shape) may lose the benefit of the immersion provided by VR headsets. In addition, our focus is on the basic aspect of VR than the complex ones. The color of ambient light suits these requirements. Our work is different from Chen et al. (2020) which used VR environments involving a diversity of visual elements, surface textures, and types of rooms. Moreover, while the experiment by Wang et al. (2020) and Huang et al. (2019) focused on the changing the color of the drink in VR (i.e., an intrinsic property of the drinks), our study dealt with the ambient color of the VR environment (i.e., an extrinsic property of the drinks). In our experiment, the sweetness of the beverage seems to be increased by the pink environment, and the orange VR environment was preferable. These findings can be compared with Wang et al. (2020)’s results showing that the intrinsic beverage color did not influence the perceived sweetness or liking of the beverage.

5.2 Conceptual model and emotional valence under the ambient colors

Participants’ answer to the associated scene in the VR environment shows that most participants’ color-taste associations were based on specific scenarios. This mainly includes foodstuff and non-food-related scenarios. According to the explanation by Spence (2019b) (see Section 2.2), people can often point to specific source object(s) that embody both color and taste, which is also in line with the associated scenarios regarding foodstuff in our experiment. The ambient color may trigger participants’ recall of real-life experiences with food, and then the association with taste would come up based on these experiences. The variety of answers indicates the association may be personal and subjective, which may also be a topic for further investigation.

Some answers about scenarios other than foodstuff could be explained by emotional mediation (Spence, 2011) (see Section 2.2) of the relation between taste and the words regarding emotion in a particular culture. Categorized by taste association, some answers such as “chemical poison”, “nursing house”, “horror movie” appeared under bitter association while “vineyard”, “club”, “children’s room” appeared under sweet association. When categorized by color, there were some negative words like “prison” for the black virtual environment, “horror” for the red virtual environment, and “depressed” for the green virtual environment. This kind of negative association observed with certain ambient colors may be in line with some recent studies that have explored negative emotions in VR (Lavoie et al., 2021; Magdin et al., 2021). These studies showed that virtual environments involving a higher level of absorption may increase negative emotional responses. Also, Elliot et al. (2007) suggested that attention must be paid to how color can act as a subtle environmental cue that has important influences on human behavior. Regarding color study in virtual environments, research work has shown that the context in a virtual environment where the ambient color was assigned can also influence emotion and valance (Lipson-Smith et al., 2020). This leads to a future question to be discussed since, in this experiment, we did not include 3D models in the scenes. Based on this, we suggest that future researchers or designers applying ambient color in VR systems can pay special attention to the emotional association the color can trigger.

5.3 Future applications

Our experiment results could contribute to the design of human-food applications, presenting the possibility of how the dieting experience can be altered by using specific environmental colors. Researchers have looked into the use of diet activities as positive psychological interventions to increase everyday happiness and well-being (Fischler, 2011; Cook et al., 2020). Moreover, existing works have explored what and how advanced technology can support healthy dieting using wearable devices and mobile apps (Dong and Biswas, 2013; Pan et al., 2019). Our work shows the possibility of using VR to change people’s taste perception, especially the sensation of sweet taste. Given the health issues caused by high levels of sugary drink intake (Essman et al., 2021; Leung et al., 2021), our findings could have applications to support healthier drinking habits. We found that modifying the ambient color in a VR environment can enhance users’ sensation of the sweet taste of the beverage. This approach provides a novel beverage-drinking experience and a cost-effective approach to changing users’ diet behavior (by providing users’ perception of sweetness with drinks that are lower in actual sugar content). Future research could look into this potential application via longitudinal experiments.

6 Limitations and future work

The first limitation of this research is that our sample population is limited to university students within a similar age group and is relatively small. To examine the topic further, a larger sample size including different age groups may be helpful taking our experiment framework as a reference. Also, in our experiments, participants did the taste ratings after receiving the stimuli at a consistent time interval. Further work is needed to check whether the same results will be obtained and whether other insights will be found when we extend or shorten the exposure time of the ambient color. Moreover, the aroma of the tea can be crucial for influencing the taste and could represent a bias for studying taste perception. However, our focus was the influence of ambient light on basic tastes, and we did not want to introduce the aroma of tea as another variable. We tried to reduce the influence of tea aroma on taste perception by covering it with lids for participants to hold in their hands to reduce the influence. Despite these limitations, we believe that our work contributes to the initial direction and provides a solid foundation on top of which further research could be conducted to better understand CMC, especially the ones evoked by ambient color in virtual environments.

Thinking more broadly, the virtual environment can be rendered in various ways and the VR HMD we used in this experiment is just one of them. In other types of virtual environments, including CAVE, the screen-based interface can also be taken into account to see if there is a difference in taste perception or not. Moreover, the only beverage we used was a neutral drink. In the future, it could be interesting to use several types of beverages and explore whether for example using beverages with a strong taste can have a greater or lesser effect.

7 Conclusion

Our research provides a solid example of using a Virtual Reality (VR) environment as an experimental platform to study the cross-modal correspondence between ambient color and taste perception. Results from the first experiment suggest the need for paying attention to the visually induced motion sickness and color recognition of people when doing color-related experiments in VR environments. Results from the second experiment demonstrated that the pink ambient color could change people’s perception of the sweet taste of a neutral drink. The black and green ambient colors are associated with bitterness, while pink and purple are linked with sweetness. Moreover, the orange ambient color is preferable for drinking beverages in VR. Our research fills a gap in cross-modal correspondence targeting ambient color in VR, providing helpful and practical insights that can be applied to human-food interaction and VR applications that involve socialization and healthy habit promotion. Its results can also form the basis for further research.

Data availability statement

The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by University Research Committee. The patients/participants provided their written informed consent to participate in this study.

Author contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication. ZW and MJ conducted this work when they were affiliated with Xi’an Jiaotong-Liverpool University.

Funding

This work is funded in part by Xi’an Jiaotong-Liverpool University (XJTLU) Key Special Fund (KSF-A-03) and XJTLU Research Development Fund (RDF-17-01-54), and the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (22KJB520038).

Acknowledgments

The authors want to thank the participants for volunteering their time to join the experiments. We also thank the reviewers for the insightful comments and suggestions that helped improve our paper.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

¹Umami is a category of taste in food (besides sweet, sour, salt, and bitter), corresponding to the flavor of glutamates, especially monosodium glutamate.

²https://hello.vrchat.com/.

³https://www.oculus.com/workrooms/.

References

Arnold, P. (2017). You better eat to survive! exploring edible interactions in a virtual reality game. Proc. 2017 CHI Conf. Ext. Abstr. Hum. Factors Comput. Syst. ’17, 206–209. doi:10.1145/3027063.3048408

PubMed Abstract | CrossRef Full Text | Google Scholar

Bangcuyo, R. G., Smith, K. J., Zumach, J. L., Pierce, A. M., Guttman, G. A., and Simons, C. T. (2015). The use of immersive technologies to improve consumer testing: The role of ecological validity, context and engagement in evaluating coffee. Food Qual. Prefer. 41, 84–95. doi:10.1016/j.foodqual.2014.11.017

CrossRef Full Text | Google Scholar

Bonato, A., Bubka, F., and Alfieri, L. (2004). Display color affects motion sickness symptoms in an optokinetic drum. Aviat. Space Environ. Med. 75, 306–311.

PubMed Abstract | Google Scholar

Cardello, A. V. (1994). Consumer expectations and their role in food acceptance. Boston, MA: Springer US, 253–297. doi:10.1007/978-1-4615-2171-6_10

CrossRef Full Text | Google Scholar

Carlson, P., Kirpes, C., Pavlik, R. A., Vance, J. M., Yin, L., Scott-Cooper, T., et al. (2011). “Comparison of single-wall versus multi-wall immersive environments to support a virtual shopping experience,” in ASME 2011 World Conference on Innovative Virtual Reality, Milan, Italy, June 27–29, 2011 (World Conference on Innovative Virtual Reality), 287–291. doi:10.1115/WINVR2011-5582

CrossRef Full Text | Google Scholar

Charles, S., Carmel, A., L., Maya, U. S., and Massimiliano, Z. (2010). Does food color influence taste and flavor perception in humans? Chemosens. Percept. 3, 68–84. doi:10.1007/s12078-010-9067-z

CrossRef Full Text | Google Scholar

Chen, Y., Huang, A. X., Faber, I., Makransky, G., and Perez-Cueto, F. J. A. (2020). Assessing the influence of visual-taste congruency on perceived sweetness and product liking in immersive vr. Foods 9, 465. doi:10.3390/foods9040465

PubMed Abstract | CrossRef Full Text | Google Scholar

Clydesdale, F. M., Gover, R., and Fugardi, C. (1992). The effect of color on thirst quenching, sweetness, acceptability and flavor intensity in fruit punch flavored beverages. J. Food Qual. 15, 19–38. doi:10.1111/j.1745-4557.1992.tb00973.x

CrossRef Full Text | Google Scholar

Cook, M., Kuntsche, S., Labhart, F., and Kuntsche, E. (2020). Do different drinks make you feel different emotions? Examination of young adolescents’ beverage-specific alcohol expectancies using the alcohol expectancy task. Addict. Behav. 106, 106375. doi:10.1016/j.addbeh.2020.106375

PubMed Abstract | CrossRef Full Text | Google Scholar

Crisinel, A.-S., and Spence, C. (2010). As bitter as a trombone: Synesthetic correspondences in nonsynesthetes between tastes/flavors and musical notes. Atten. Percept. Psychophys. 72, 1994–2002. doi:10.3758/APP.72.7.1994

PubMed Abstract | CrossRef Full Text | Google Scholar

Crisinel, A.-S., and Spence, C. (2009). Implicit association between basic tastes and pitch. Neurosci. Lett. 464, 39–42. doi:10.1016/j.neulet.2009.08.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Crofton, E., Murray, N., and Botinestean, C. (2021). Exploring the effects of immersive virtual reality environments on sensory perception of beef steaks and chocolate. Foods 10, 1154. doi:10.3390/foods10061154

PubMed Abstract | CrossRef Full Text | Google Scholar

Delarue, J., Brasset, A.-C., Jarrot, F., and Abiven, F. (2019). Taking control of product testing context thanks to a multi-sensory immersive room. a case study on alcohol-free beer. Food Qual. Prefer. 75, 78–86. doi:10.1016/j.foodqual.2019.02.012

CrossRef Full Text | Google Scholar

Dong, B., and Biswas, S. (2013). “Wearable diet monitoring through breathing signal analysis,” in 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, 03-07 July 2013, 1186–1189. doi:10.1109/EMBC.2013.6609718

PubMed Abstract | CrossRef Full Text | Google Scholar

Elliot, A. J., Maier, M. A., Moller, A. C., Friedman, R., and Meinhardt, J. (2007). Color and psychological functioning: The effect of red on performance attainment. J. Exp. Psychol. General 136, 154–168. doi:10.1037/0096-3445.136.1.154

PubMed Abstract | CrossRef Full Text | Google Scholar

Essman, M., Stoltze, F. M., Carpentier, F. D., Swart, E. C., and Taillie, L. S. (2021). Examining the news media reaction to a national sugary beverage tax in South Africa: A quantitative content analysis. BMC public health 21, 454–514. doi:10.1186/s12889-021-10460-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Fischler, C. (2011). Commensality, society and culture. Soc. Sci. Inf. 50, 528–548. doi:10.1177/0539018411413963

CrossRef Full Text | Google Scholar

Gallace, A., Ngo, M. K., Sulaitis, J., and Spence, C. (2012). “Multisensory presence in virtual reality: Possibilities & limitations,” in Multiple sensorial media advances and applications: New developments in MulSeMedia (Pennsylvania, United States: IGI Global), 1–38. doi:10.4018/978-1-60960-821-7.ch001

CrossRef Full Text | Google Scholar

Gotow, N., Skrandies, W., Kobayashi, T., and Kobayakawa, T. (2021). Traditional Japanese confection overseas: Cultural difference and retronasal aroma affect flavor preference and umami perception. Food Qual. Prefer. 92, 104204. doi:10.1016/j.foodqual.2021.104204

CrossRef Full Text | Google Scholar

Grassini, S., Laumann, K., de Martin Topranin, V., and Thorp, S. (2021). Evaluating the effect of multi-sensory stimulations on simulator sickness and sense of presence during hmd-mediated vr experience. Ergonomics 64, 1532–1542. doi:10.1080/00140139.2021.1941279

PubMed Abstract | CrossRef Full Text | Google Scholar

Gusev, D. A., Eschbach, R., Westin, T., and Yong, J. (2018). Motion-sickness-related aspects of inclusion of color-deficient observers in virtual reality. Int. J. Child Health Hum. Dev. 11, 177–181.

Google Scholar

Gusev, D. A., Whittinghill, D. M., and Yong, J. (2016). “A simulator to study the effects of color and color blindness on motion sickness in virtual reality using head-mounted displays,” in Mobile and wireless technologies 2016. Editors K. J. Kim, N. Wattanapongsakorn, and N. Joukov (Singapore: Springer Singapore), 197–204.

CrossRef Full Text | Google Scholar

Halabi, O., and Saleh, M. (2021). Augmented reality flavor: Cross-modal mapping across gustation, olfaction, and vision. Multimed. Tools Appl. 1, 36423–36441. doi:10.1007/s11042-021-11321-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Harrar, V., Piqueras-Fiszman, B., and Spence, C. (2011). There’s more to taste in a coloured bowl. Perception 40, 880–882. doi:10.1068/p7040

PubMed Abstract | CrossRef Full Text | Google Scholar

Huang, F., Huang, J., and Wan, X. (2019). Influence of virtual color on taste: Multisensory integration between virtual and real worlds. Comput. Hum. Behav. 95, 168–174. doi:10.1016/j.chb.2019.01.027

CrossRef Full Text | Google Scholar

Johnson, J., and Clydesdale, F. M. (1982). Perceived sweetness and redness in colored sucrose solutions. J. Food Sci. 47, 747–752. doi:10.1111/j.1365-2621.1982.tb12706.x

CrossRef Full Text | Google Scholar

Keast, R. S., and Breslin, P. A. (2003). An overview of binary taste–taste interactions. Food Qual. Prefer. 14, 111–124. doi:10.1016/S0950-3293(02)00110-6

CrossRef Full Text | Google Scholar

Kerruish, E. (2019). Arranging sensations: Smell and taste in augmented and virtual reality. Senses Soc. 14, 31–45. doi:10.1080/17458927.2018.1556952

CrossRef Full Text | Google Scholar

Kim, H. K., Park, J., Choi, Y., and Choe, M. (2018). Virtual reality sickness questionnaire (vrsq): Motion sickness measurement index in a virtual reality environment. Appl. Ergon. 69, 66–73. doi:10.1016/j.apergo.2017.12.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Knöferle, K., and Spence, C. (2012). Crossmodal correspondences between sounds and tastes. Psychon. Bull. Rev. 1, 992–1006. doi:10.3758/s13423-012-0321-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Koch, C., and Koch, E. C. (2003). Preconceptions of taste based on color. J. Psychol. 137, 233–242. doi:10.1080/00223980309600611

PubMed Abstract | CrossRef Full Text | Google Scholar

Kuliga, S., Thrash, T., Dalton, R., and Hölscher, C. (2015). Virtual reality as an empirical research tool — Exploring user experience in a real building and a corresponding virtual model. Comput. Environ. Urban Syst. 54, 363–375. doi:10.1016/j.compenvurbsys.2015.09.006

CrossRef Full Text | Google Scholar

Lavoie, R., Main, K., King, C., and King, D. (2021). Virtual experience, real consequences: The potential negative emotional consequences of virtual reality gameplay. Virtual Real. 25, 69–81. doi:10.1007/s10055-020-00440-y

CrossRef Full Text | Google Scholar

Ledoux, T., Nguyen, A. S., Bakos-Block, C., and Bordnick, P. (2013). Using virtual reality to study food cravings. Appetite 71, 396–402. doi:10.1016/j.appet.2013.09.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Leung, C. Y., Abe, S. K., Sawada, N., Ishihara, J., Takachi, R., Yamaji, T., et al. (2021). Sugary drink consumption and risk of kidney and bladder cancer in Japanese adults. Sci. Rep. 11, 21701–21709. doi:10.1038/s41598-021-01103-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Lipson-Smith, R., Bernhardt, J., Zamuner, E., Churilov, L., Busietta, N., and Moratti, D. (2020). Exploring colour in context using virtual reality: Does a room change how you feel? Virtual Real. 25, 631–645. doi:10.1007/s10055-020-00479-x

CrossRef Full Text | Google Scholar

Maga, J. A. (1974). Influence of color on taste threshold. Chem. Senses 1, 115–119. doi:10.1093/chemse/1.1.115

CrossRef Full Text | Google Scholar

Magdin, M., Balogh, Z., Reichel, J., Francisti, J., Koprda, Š., and György, M. (2021). Automatic detection and classification of emotional states in virtual reality and standard environments (lcd): Comparing valence and arousal of induced emotions. Virtual Real. 1, 1029–1041. doi:10.1007/s10055-021-00506-5

CrossRef Full Text | Google Scholar

Moser, C., and Tscheligi, M. (2013). “Playful taste interaction,” in IDC ’13: Proceedings of the 12th International Conference on Interaction Design and Children, New York, NY, 24 June 2013 (New York: Association for Computing Machinery), 340–343. doi:10.1145/2485760.2485828

CrossRef Full Text | Google Scholar

Narumi, T., Nishizaka, S., Kajinami, T., Tanikawa, T., and Hirose, M. (2011). “Augmented reality flavors: Gustatory display based on edible marker and cross-modal interaction,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, NY, USA, 07 May 2011 (Association for Computing Machinery), 93–102. CHI ’11. doi:10.1145/1978942.1978957

CrossRef Full Text | Google Scholar

Nishizawa, M., Jiang, W., and Okajima, K. (2016). “Projective-ar system for customizing the appearance and taste of food,” in Proceedings of the 2016 Workshop on Multimodal Virtual and Augmented Reality, New York, NY, USA, 16 November 2016 (Association for Computing Machinery). MVAR ’16. doi:10.1145/3001959.3001966

CrossRef Full Text | Google Scholar

Oberfeld, D., Hecht, H., Allendorf, U., and Wickelmaier, F. (2009). Ambient lighting modifies the flavor of wine. J. Sens. Stud. 24, 797–832. doi:10.1111/j.1745-459X.2009.00239.x

CrossRef Full Text | Google Scholar

Oberfeld, D., and Hecht, H. (2011). Fashion versus perception: The impact of surface lightness on the perceived dimensions of interior space. Hum. Factors 53, 284–298. doi:10.1177/0018720811407331

PubMed Abstract | CrossRef Full Text | Google Scholar

Pan, S., Ren, X., Brombacher, A., and Vos, S. (2019). “Designing technology to encourage healthy eating at work,” in Proceedings of the 9th International Conference on Digital Public Health, New York, NY, USA, 20 November 2019 (Association for Computing Machinery), 131. DPH2019. doi:10.1145/3357729.3357759

CrossRef Full Text | Google Scholar

Parise, C., and Spence, C. (2013). “Audiovisual cross-modal correspondences in the general population,” in The Oxford handbook of synesthesia (Oxford, United Kingdom: Oxford University Press).

CrossRef Full Text | Google Scholar

Piqueras-Fiszman, B., and Spence, C. (2014). Colour, pleasantness, and consumption behaviour within a meal. Appetite 75, 165–172. doi:10.1016/j.appet.2014.01.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Piqueras-Fiszman, B., and Spence, C. (2012). The influence of the color of the cup on consumers’ perception of a hot beverage. J. Sens. Stud. 27, 324–331. doi:10.1111/j.1745-459X.2012.00397.x

CrossRef Full Text | Google Scholar

Slater, M., and Wilbur, S. (1997). A framework for immersive virtual environments (five): Speculations on the role of presence in virtual environments. Presence. (Camb). 6, 603–616. doi:10.1162/pres.1997.6.6.603

CrossRef Full Text | Google Scholar

So, R. H., and Yuen, S. (2007). “Comparing symptoms of visually induced motion sickness among viewers of four similar virtual environments with different color,” in Virtual reality (Berlin, Germany: Springer Berlin Heidelberg), 386–391. doi:10.1007/978-3-540-73335-5_42

CrossRef Full Text | Google Scholar

Spence, C., and Carvalho, F. M. (2020). The coffee drinking experience: Product extrinsic (atmospheric) influences on taste and choice. Food Qual. Prefer. 80, 103802. doi:10.1016/j.foodqual.2019.103802

CrossRef Full Text | Google Scholar

Spence, C. (2011). Crossmodal correspondences: A tutorial review. Atten. Percept. Psychophys. 73, 971–995. doi:10.3758/s13414-010-0073-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Spence, C., and Levitan, C. A. (2021). Explaining crossmodal correspondences between colours and tastes. i-Perception 12, 204166952110182. doi:10.1177/20416695211018223

PubMed Abstract | CrossRef Full Text | Google Scholar

Spence, C. (2019a). Multisensory experiential wine marketing. Food Qual. Prefer. 71, 106–116. doi:10.1016/j.foodqual.2018.06.010

CrossRef Full Text | Google Scholar

Spence, C. (2019b). On the relationship(s) between color and taste/flavor. Exp. Psychol. 66, 99–111. doi:10.1027/1618-3169/a000439

PubMed Abstract | CrossRef Full Text | Google Scholar

Spence, C. (2016). Oral referral: On the mislocalization of odours to the mouth. Food Qual. Prefer. 50, 117–128. doi:10.1016/j.foodqual.2016.02.006

CrossRef Full Text | Google Scholar

Spence, C., and Piqueras-Fiszman, B. (2016). “Food color and its impact on taste/flavor perception,” in Multisensory flavor perception (Amsterdam, Netherlands: Elsevier), 107–132.

CrossRef Full Text | Google Scholar

Spence, C., Velasco, C., and Knoeferle, K. (2014). A large sample study on the influence of the multisensory environment on the wine drinking experience. Flavour 3, 8–12. doi:10.1186/2044-7248-3-8

CrossRef Full Text | Google Scholar

Spence, C., and Velasco, C. (2018). On the multiple effects of packaging colour on consumer behaviour and product experience in the ‘food and beverage’ and ‘home and personal care’ categories. Food Qual. Prefer. 68, 226–237. doi:10.1016/j.foodqual.2018.03.008

CrossRef Full Text | Google Scholar

Stein, B. E., and Meredith, M. A. (1993). The merging of the senses. Massachusetts, United States: MIT Press.

Google Scholar

Stelick, A., Penano, A. G., Riak, A. C., and Dando, R. (2018). Dynamic context sensory testing–a proof of concept study bringing virtual reality to the sensory booth. J. Food Sci. 83, 2047–2051. doi:10.1111/1750-3841.14275

PubMed Abstract | CrossRef Full Text | Google Scholar

Sugimori, E., and Kawasaki, Y. (2022). Cross-modal correspondence between visual information and taste perception of bitter foods and drinks. Food Qual. Prefer. 98, 104539. doi:10.1016/j.foodqual.2022.104539

CrossRef Full Text | Google Scholar

Unity (2019). Global illumination. Available at: https://docs.unity.cn/2019.1/Documentation/Manual/GlobalIllumination.html (Accessed November 24, 2022).

Google Scholar

von Castell, C., Hecht, H., and Oberfeld, D. (2018). Which attribute of ceiling color influences perceived room height? Hum. Factors 60, 1228–1240. doi:10.1177/0018720818789524

PubMed Abstract | CrossRef Full Text | Google Scholar

Wan, X., Woods, A. T., van den Bosch, J. J., McKenzie, K. J., Velasco, C., and Spence, C. (2014). Cross-cultural differences in crossmodal correspondences between basic tastes and visual features. Front. Psychol. 5, 1365. doi:10.3389/fpsyg.2014.01365

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, Q. J., Barbosa Escobar, F., Alves Da Mota, P., and Velasco, C. (2021). Getting started with virtual reality for sensory and consumer science: Current practices and future perspectives. Food Res. Int. 145, 110410. doi:10.1016/j.foodres.2021.110410

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, Q. J., Meyer, R., Waters, S., and Zendle, D. (2020). A dash of virtual milk: Altering product color in virtual reality influences flavor perception of cold-brew coffee. Front. Psychol. 11, 595788. doi:10.3389/fpsyg.2020.595788

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, Q. J., Mielby, L. A., Junge, J. Y., Bertelsen, A. S., Kidmose, U., Spence, C., et al. (2019). The role of intrinsic and extrinsic sensory factors in sweetness perception of food and beverages: A review. Foods 8, 211. doi:10.3390/foods8060211

PubMed Abstract | CrossRef Full Text | Google Scholar

Wobbrock, J. O., Findlater, L., Gergle, D., and Higgins, J. J. (2011). “The aligned rank transform for nonparametric factorial analyses using only anova procedures,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, NY, USA, 07 May 2011 (Association for Computing Machinery), 143–146. CHI ’11. doi:10.1145/1978942.1978963

PubMed Abstract | CrossRef Full Text | Google Scholar