Yashu Wang
Yiwen Wang- School of Arts and Communication, Beijing Normal University, Beijing, China
As a new and rapidly emerging cross-disciplinary research field, neurocinematics focuses on movie research from an empirical perspective, adopting functional magnetic resonance imaging (fMRI), and other cognitive neuroscience technologies as well as theoretical methods. By verifying and exploring relevant film concepts, neurocinematics tries to establish a scientific basis for the movie theory and better understand frontier subjects in movie studies. We designed this experiment to detect audiences' brain activity when watching movies and verify the manipulation power of narrative film. We selected the shower murder scene in Hitchcock's Psycho as the experimental material. The results of the experiment showed that the trends of the audiences' brain activity were almost consistent with the specific movie plots. By observing audiences' brain activity while watching the movie, the experiment verified the specific effect of Hitchcock's set-up of suspense and explored the neurocognitive brain mechanisms behind the suspense effect.
Background
Media and cinema studies have always struggled to understand how cinema influences and manipulates human emotion (Bondebjerg, 2014; Raz and Hendler, 2014). From the work of the gestalt analysis performed by Rudolf Arnheim to the development of Montage Theory by Sergei Eisenstein, and from the application of photography, editing, and other technologies to the technological revolution of 3D and virtual reality, all of these efforts have been made theoretically and practically (Cutting et al., 2011). Based on the abundant existing research, more empirical, quantitative, and interdisciplinary evidence is currently required. When measuring viewers' perceptions of a movie, the box office is affected by various factors, and it cannot reflect the complicated feelings of the audience (Kwak and Zhang, 2011). Behavioral measurements such as questionnaire surveys and interviews are adopted in empirical studies of cinema, which have become essential in these studies. However, inappropriate sampling, poor factor structure, or low internal consistency reliability may reduce the statistical reliability and validity, and subjects may claim more favorable behavior to please the interviewer or comply with accepted norms (Hinkin, 1995; Chestnutt et al., 2004). Most of all, these methods lack the synchronous and dynamic capture of the audience's viewing psychology. The results are obtained after the film viewing activity had ended. Blood pressure, heart rate, and eye movement provide synchronous physiological data of movie-watching reactions to a certain extent. However, they are the external physiological manifestations of cognitive results and cannot explain the cognitive process or its mechanism in-depth when responding to movies. A deeper and more comprehensive understanding of movies' impact on audiences requires introducing new research methods.
Movies are multimodal, providing integrating audio-visual information, logical processes, and emotional experience. The process of humans watching movies is similar to the process of information processing in the daily environment. Understanding how the brain accepts and responds to movies will help understand how the brain perceives the real world (Dudai, 2012). Since the beginning of the millennium, increasing numbers of cognitive neuroscientists have been using movie clips to study the human brain, and the interdisciplinary links between movies and the human brain have become increasingly close. Under the above circumstances, the interdisciplinary interactions between cognitive neuroscience and cinematography provide us with a suitable method (Motz, 2013).
Cognitive neuroscience is a fundamental biological science that combines theories and experimental evidence from neuropsychology, neuroscience, and computational models. Functional magnetic resonance imaging (fMRI) measures neural activity with a high spatial resolution and is the only non-invasive and precision-oriented method for studying advanced brain functions. By measuring the blood oxygen response in subjects' brains while watching a movie, fMRI can accurately detect and locate the brain regions that generate activity during complex stimuli and determine which brain regions are called upon and to what extent (Logothetis et al., 2001).
As a new and interdisciplinary field, neurocinematic studies mainly adopt empirical research, and experiment as common means. Researchers use fMRI technology and theories of cognitive neuroscience to perform empirical research on movies to verify and explore related topics in the field of film, to attempt to establish a scientific foundation for film theory, and to discuss the discipline's frontier topics in theory and practice (Hasson et al., 2008a,b). In recent years, the interdisciplinary cooperation between cinematic and cognitive neuroscience has gradually become a trendy field worldwide (Dmochowski et al., 2014; Bondebjerg, 2017). However, there are still some gaps regarding specific interdisciplinary issues (Stadler, 2018). A typical example is that cognitive neuroscientists pay more attention to brain structures and functions or psychotherapy than movie researchers, who show increased interest in movie audiences' reactions. The former approach treats movies as experimental materials for brain research, while the latter approach uses brain knowledge and technology to understand better movies themselves (Willems et al., 2011; Plantinga, 2012; Bondebjerg, 2014; Francuz and Mendyk, 2014; Pehrs et al., 2014; Lahnakoski et al., 2017). Finally, the research team of Uri Hasson from the Weizmann Institute of Science in Israel brought the interdisciplinary vision into reality.
Hasson's team used a clip of the first 30 min of the famous film The Good, The Bad, and The Ugly as their experimental material. They recruited five participants to watch the clip in a functional MRI scanner without additional experimental tasks. Through naturalistic viewing, the participants' direct brain responses had been captured by the scanner. Compared to traditional fMRI experiments, which usually use single stimuli such as a single picture or word and ask subjects to do responding or rating tasks during the scanning process, Hasson's experimental designs replicated the audience's daily natural movie-watching mode to some extent, without any human intervention in the watching behavior. The experiment can be carried out under a continual and natural, therefore relatively ecological condition. Inter-subject correlation analysis (ISC) was a critical component of Hasson's research. By comparing the consistency of different subjects' brain activity when watching the same movie, the influence of movies on an audience can be generally and universally discussed. The results showed that while the five subjects watched the movie, they showed similar brain activities at a high level, suggesting that movies can control the human brain to some extent (Hasson et al., 2004).
Hasson's research moved the combination of cinematics and cognitive neuroscience from theory into a period of actuation and development. Despite Hasson's research, empirical neurocinematic studies of particular problems of movies initiated by movie researchers are still rare at present. In the field of cinematics, to make movies attract larger audiences has always been a concern of movie-makers (Rooney and Hennessy, 2013). At the same time, it is also one of the topics that movie researchers focus on (Bordwell, 2010). According to narrative transportation theory, attractive narrations suppress audiences' perception of surroundings beyond the screen and make audiences more immersed in the story world, even take the story for real (Gerrig, 1993). Suspense is one of the factors linked to increased transportation (Tal-Or and Cohen, 2010). As a fundamental human emotion, suspense plays a pivotal role in attracting an audience's attention (Cheong and Young, 2008; Lehne and Koelsch, 2015). On the one hand, it usually means some terrible events will happen, but on the other hand, a suspense moment is also the moment when terrible things have not happened yet. Therefore, when watching suspense movies, audiences hold strong emotions like fear, hate, or anxiety and need to activate complex cognitive mechanisms, such as prediction, anticipation, or moral judgment (Comisky and Bryant, 1982; Naab and Sukalla, 2019). As to study audience involvement in movies, suspense is a suitable human emotion.
One of the definitions of suspense is anxiety arising from an uncertain situation, and no one could produce that response in film audiences better than Alfred Hitchcock (Adair, 2002). In world movie history, Hitchcock has been regarded as a master who can trigger fear inside of audiences. According to David Bordwell, Hitchcock's movies are some of the most suitable materials for film researchers to study since they enjoy certain accessibility through their stylistic and thematic obviousness to audiences (Belton, 2003). Francois Truffaut, the famous French director, and film theorist, once commented on Hitchcock, saying that “he devoted himself to making the audience afraid. Let the audience find the feeling of childhood when playing hide-and-seek in a quiet house and hiding behind furniture to be caught” (Truffaut, 1983).
Among Hitchcock's many works, Psycho broke taboos with its iconic shower-murder scene and its rejection of the norms of good taste (Banash, 2015). In movie history, Psycho is also the unfading classic and has still been repeatedly analyzed or imitated to this day. Released in the 1960s, this movie was a huge success and the supreme audience-identification film (Forbes, 1969; Thomson, 2009). Compared with horror movies that use blood, screams, and ghosts to scare the audience deliberately, Psycho created a psychological and personal terror that has left an indelible impression on generations of audiences and has also become a cultural fantasy about the overwhelming powers of cinema (Banash, 2015). Therefore, the shower murder scene from Psycho is the ideal material for this experiment.
From the above, this paper draws on Hasson's team's method to study the brain activity consistency of subjects while watching Hitchcock's Psycho from the perspective of cinematics, and it preliminarily discusses the movie's suspense effect. On this basis, the movie and its suspense effect will be analyzed from a cognitive perspective. fMRI experiment and questionnaire survey will be combined to make these following hypothesis:
a. From a whole-brain perspective, the narrative, carefully structured suspense film clip has a synchronous and consistent effect on brain activity beyond multiple subjects. Compared with non-fiction or documentary movies, well-designed narrative movies have more robust control over the audiences.
b. From the functional brain regions' perspective, visual and auditory regions' brain activity characteristics may be related to the movie plot and maybe more active at critical plot points. Visual brain regions may be influenced by the design of movie images such as editing, the lens view, and the camera movement. The auditory brain may be influenced by the sound design in the movie, especially the music.
c. From the perspective of subjective memory and emotional arousal, the audience's memory and emotional response to the screenshot images may share the same characteristic to the brain activity in fMRI scanning. The performance of the memory and the emotion of the audience will be more active in the images of critical plots but less active in the images of transitional or unimportant plots.
Materials and Methods
Stimuli
The experimental material consists of two video clips, one of which is cut from the shower murder scene in Psycho. The video's length is 5 min and 42 s, with no dialogue except in the last shot. This video is the leading research object of this experiment. The starting and ending frames of the experimental video are shown in Figure 1. The control video is a natural street scene outside the art school building of Beijing Normal University. Compared with the first video from Psycho, the second video does not have a specific plot or narration. It is a simulation of daily life, and the participants' viewing is closer to viewing in daily status. Both videos are in black and white. The aspect ratio and duration of the control video are the same as those in the clip of Psycho, as shown in Figure 2.
Figure 1. Beginning and ending shots of the Psycho clip.
Figure 2. Clip of the street scene.
Participants and Procedures
Participants
In total, 10 participants were recruited for the experiment, all of whom were college students aged between 21 and 25 years old. Five participants were male, and five participants were female, and all participants were right-handed and were not the movie or art majors. None of the participants had seen Psycho before, and they did not know the specific content of the experimental materials. All the participants have signed the informed consent forms before the experiment.
The fMRI Scanning Procedure
The study procedure consisted of two phases. In the first phase, fMRI was used to scan the subjects' brain activity while watching the movie and study the correlation of brain activity among the different subjects. During the experiment, the subjects laid flat in an fMRI scanner and viewed both video clips in a natural state. The videos were presented on a computer screen, and the sound was provided by headphones designed for the scanner. The subjects had no additional task requirements other than watching the videos. To eliminate errors caused by the sequence of the two video clips, we balanced the video viewing sequence among the subjects; five subjects watched the movie clip first and then watched the natural scene, while the other five watched the clips in the opposite order. During the whole process, the subjects' brain activities were scanned and recorded. After that, we calculated the ISC among ten subjects to obtain the average brain response during the watching process.
The fMRI experiment was conducted at the Institute of Brain and Cognitive Science of Beijing Normal University and the State Key Laboratory of Cognitive Neuroscience and Learning. It was supported by the review committee at the State Key Laboratory of Cognitive Neuroscience and Learning of Beijing Normal University.
Brain data acquisition was performed using a Siemens Trio Tim 3.0T scanner (Siemens Medical System, Erlangen, Germany) with a 12-channel phased-array head coil in the Imaging Center for Brain Research at the Beijing Normal University. The fMRI data were obtained using an echo-planar imaging sequence with the following parameters: repetition time (TR)/echo time (TE) = 2,000 ms/30 ms; flip angle (FA) = 90°; field of view (FOV) = 200 × 200 mm2; matrix = 64 × 64; slices = 33; thickness = 3.5 mm; voxel size = 3.1 × 3.1 × 3.5 mm3; gap = 0.7 mm; and 171 volumes. T1-weighted data were acquired using sagittal 3D magnetization prepared rapid gradient echo sequences. The sequence parameters were as follows: TR/TE = 2,530 ms/3.39 ms; FA = 7°; FOV = 256 × 256 mm2, matrix = 256 × 192; number of slices = 144; thickness = 1.3 mm; and voxel size = 1 × 1.3 × 1.3 mm3.
A series of preprocessing procedures were performed for the fMRI data to reduce non-neural noise by using Statistical Parametric Mapping software (SPM8; www.fil.ion.ucl.ac.uk/spm) and the Data Processing Assistant for Resting-State fMRI (DPARSF) (Yan and Zang, 2010), including slice time correction, head motion correction, a temporal band-pass filter (0.01–1.25 Hz), detrending, normalization to the Montreal Neurological Institute (MNI) space based on the T1 image, and spatial smoothing. Then, for each participant, the preprocessed time course of each voxel within a gray matter mask was extracted for the following analyses.
To quantitatively estimate subjects' involvement in the movie based on the brain fMRI data, we calculated the inter-subject correlation (ISC) for each voxel. Specifically, for a given voxel in the brain, we calculated Pearson's correlation coefficient between the time course of a subject and the averaged time course of all the other subjects. Overall, there were 45 unique pairwise comparisons between the 10 subjects watching the same clip. Then, we averaged the Pearson's correlation coefficient across 10 subjects to obtain a whole-brain activity consistency map for the movie and the natural condition. By calculating the ISC of 10 subjects, we can examine the intersubject dimension of film watching and determine whether the audience has a similar cognitive process.
The Behavioral Test Procedure
The second phase was a behavioral test that was a supplement to the first part of the brain imaging experiment. After the brain imaging scans were completed, the subjects entered another room to take an image recognition and behavioral test related to the video clips. The materials for the behavior test were programmed with E-Prime software and were presented on a computer screen. The subjects were asked to press a key to answer the questions. When randomly presented with an image, the subjects first judged whether they had seen it during the previous fMRI scan. If they had seen it, they pressed button 1; if had not seen it, they pressed button 0. Since the emotional activation is a key to audience empathy and has been used to measure cognitive effects of movie narrations (Raz and Hendler, 2014; Gruskin et al., 2019), we then asked subjects to rate the emotional intensity caused by the current image on a 5-point scale, with “1” meaning no suspense feeling and “5” meaning a strong suspense feeling. The viewed images were from the video clip of Psycho shown in the previous fMRI scan. We captured one screenshot every 5 s. After removing duplicate images, 32 images were used for the test, as shown in Figure 3. To ensure the homogeneity of the two groups of images, 16 images (Figure 4) that were also screenshots from Psycho that the subjects had not seen during the previous fMRI scan were included. The subjects' memory and emotional response to previous experimental clips are measured to clarify the perception further through image recognition.
Figure 3. The sample of video screenshots that were shown during fMRI scanning.
Figure 4. The sample of video screenshots that were not shown during fMRI scanning.
Results and Discussion
Scanning Data by fMRI
Consistent Spatial Distribution of Brain Activity
By the fMRI scanning and the method of ISC, we obtained the average brain activity of subjects while watching the experiment clips. Figure 5 shows the consistent brain activity areas (Figure 5A) when the subjects watched the clip of Psycho and the consistent brain activity areas (Figure 5B) when the subjects watched the street scene (r > 0.1). When watching the movie clips, brain activity areas were larger than those of the natural scenes. As seen from the figure, the activity in the brain's areas caused by the movie clips was more intense than those caused by the natural scene. Compared with the usual daily scene, a delicately constructed movie clip has generated a more consistent cognitive process pattern.
Figure 5. (A) Arousal in brain regions while watching Psycho clip. (B) Arousal in brain regions while watching street scene clip. The colors indicate areas of consistent brain activity and the closer the color to red, the higher the activity's consistency.
According to the brain scanning result in Figure 5A, the bilateral superior temporal cortex, lateral visual cortex, fusiform, and frontal eye fields were highly activated during the Psycho clip watching. These brain regions are involved in visual and auditory information processing and are related to face recognition and eye movement (Gross and De Schonen, 1992; Owen et al., 1998; Grill-Spector and Sayres, 2008). However, during the street scene watching, those areas were little or not activated. This result suggests that fictional narration and construction of suspense lead audiences to a higher cognitive level and make them engaged in more positive perception. Audiences are more active in the aspect of visual, auditory processing, as well as attention and object recognition in movie watching than natural vision.
Changes in Brain Activity Signals Over Time
Figure 6 shows the brain activity signals of 10 subjects over time as they watched Psycho clip (Figure 6A). The horizontal axis is the timestamp of the experimental movie clips, and the vertical axis is the activity intensity of the significantly consistent areas; the larger the vertical axis value, the higher the activity intensity. Since the fMRI scanner works by scanning every 2 s, this experiment's scanning obtained 171 value points in all during Psycho clip watching process. The 10 thinner color curves in the figure show the brain activity of 10 subjects. The brain activity of 10 subjects shows a correlated trend during the movie-watching process. The bold red curve in the figure represents the average brain activity curve of the 10 subjects, shown in Figure 6B. According to the distribution of the high and low points of the curve in Figure 6B, the time stamp can be roughly divided into the following sections of durations of various lengths, and the corresponding narrative plots in the experimental video clip can be found.
Figure 6. (A) The whole-brain activity signals of ten participants while watching Psycho clip. (B) Average whole-brain activity signal of participants while watching Psycho clip.
Section A (point 1 to point 30, corresponding to 1–60 s), which includes the first high point and long duration, is the section with the most prolonged duration of high-intensity brain activity in the whole video clip, corresponding to the peeping in the movie clip. Norman's peeping makes the brain signals of the audience active, which is precisely what Hitchcock himself wanted to achieve. Joseph Stefano, the screenwriter of Psycho, recalled how Hitchcock enthused over the chance to manipulate sound to heighten audience involvement and to implicate them, with Norman Bates, as voyeurs. In the screenplay, Stefano wrote under the direction of Hitchcock: “The SOUNDS come louder as if we too had our ears pressed against the wall” (Rebello, 2004). Meanwhile, no one will ever forget that huge eye in the close-up, which is a disconcerting mixture of desire and intrigue.
Section B (point 31 to point 89, corresponding to 1–2 min and 58 s) contains a series of high points, each of which has a short duration but roughly equal intervals. The fluctuation of the curve is basically at the same level. According to the narration storyline, section B is the passage from the end of peeping to the beginning of shower murder. It is a transitional paragraph in the whole clip's narration; audiences' brain activity shows a random, irregular, and free trend. Besides, the visual and auditory design in this paragraph also has a foreshadowing characteristic. We have seen a lot of moving lens which contains information of surroundings instead of one specific character or object. The soundtrack for this section is also dominated by low and blunt bass, creating an atmosphere of uncertainty. Just like the word of Hitchcock himself, “the essential fact is, to get real suspense, you must have information” (Adair, 2002). Brain signals in Section B indicate that the audiences are invoking cognition to process the information, to understand and predict what will happen next.
Section C (point 90 to point 129, corresponding to 2 min and 58 s to ~4 min and 18 s). This section has the highest activity intensity in the whole clip and has reached to the peak intensity (1.2896). This section corresponds to the famous shower murder scene, and this is both stunningly unexpected and a logical release of the pressures built up in the long, sustained overture, and also is the climax of this clip. As a physiological and psychological result, the brain's signal intensity increased with Marion's sudden terrible scream and the simultaneous sound of a violin, the knife plunged at the poor girl, and the signal intensity got to the top of this clip at the same time. This signal movement indicates the power of murder. Shown in quick cuts and a terrific soundtrack, the bathroom is hell.
Section D (point 130 to the final point, corresponding to 4 min and 18 s to the end of the clip). Compared with the first three sections, several high-strength points and curve trends in this section do not show apparent characteristics. This section corresponds to the death of Marion and the epilog of this killing. During this section, the brain signals got weaker than the climax, but they still showed up and down changes and fluctuations. This suggests that the audiences' brain is actively processing new information, despite having just witnessed an unexpected murder and experienced severe cognitive and psychological shock. For instance, as the camera moves from the bathroom to the bedroom, the audience will see something wrapped in the newspaper next to the bed. What is that? What is that for? Does this have anything to do with the murder? All the questions are waited to be answered. The brain is going to be activated.
In summary, the brain activity of the subjects has obvious ups and downs during the whole movie-watching process, and these ups and downs correspond to different stages of plot development in the film's content, in line with Hitchcock's expectations of the effect of the movie. By the method of ISC, we found that audiences' reactions can be manipulated by the constructed narrative and audio-visual design of the movie. This consistency in brain activity is also evidence that film can reach a resonance between different people.
Activity of Brain Regions Associated With Visual and Auditory Functions
Visual Brain Activity
After discussing the whole brain regional data, we extracted the brain signals from visual and auditory regions, including the bilateral superior temporal cortex, lateral visual cortex, fusiform, and frontal eye fields. As shown in Figure 7, in terms of spatial arousal of brain activity, there was arousal in all of these brain regions, indicating that in the process of primary audio-visual processing, the film has an obvious impact on the subjects. According to the usual way of grouping high and low values in statistics, data of brain activity signals were grouped into high and low values. The mean of the high-value visual activity group is 0.7428, and the mean of the low-value visual activity group is −0.7798. The mean of the high-value auditory activity group is 0.7267, and which of the low-value auditory group is −0.7537. The independent sample T-test was performed. As in Figure 8, the result showed that p < 0.05, indicating that there were significant differences between the high-value and the low-value group both in the visual and auditory brain activities.
Figure 7. The arousal condition of visual and auditory brain regions during Psycho clip watching.
Figure 8. Statistical results of data from visual and auditory brain regions while watching Psycho clip. We have arranged all values from the largest to the smallest, according to the convention of statistical analysis, the high-value group is of the first 27%, and the low-value group is after 72%.
Figure 9 shows the time intervals with high and low visual activity values and the corresponding images. The plot contents corresponding to the time intervals with high visual activity values are Norman's peeping, calculation of money, flushing paper scraps in the toilet, shower murder, dead eye in a close-up, and the newspaper by the bed. The plot contents corresponding to the time intervals with low visual activity values are Norman's pondering, Norman walking through the corridor, Norman sitting in the living room, Marion taking a shower, Marion falling after the murder, and Marion's dead eye in a zoom-out.
Figure 9. (A) Average activity signals of visual brain regions over time. (B) Time intervals with high and low visual activity values and corresponding images.
The visual brain activity signal values are high when important events happened, such as the murder and peeping, as well as when key objects, such as the toilet bowl and the newspaper, are highlighted in the picture. In transitional scenes, such as when Norman is walking through the corridor or Marion is taking a shower, the visual brain activity values of the subjects are at a low level. The following discussions can be performed after taking the design of editing, scenes, and camera movements into consideration.
First, from the perspective of editing, the rapidly edited shots all led to a higher level of visual activity. As shown in Figure 9, the peak value group of the whole curve, between 3′40′′ and 3′50′′, is at the climax of the shower murder. In total, 16 shots are used in just 10 s, and the average length of each shot is <1 s. High-speed, dense, and coherent editing keeps the visual brain areas highly excited as they receive rich visual stimulation from the film.
Second, from the perspective of the lens view, the visual brain regions are more active with close-up shots than in a wide scene. For example, the close-up shots of Norman's peeping eye (0′32′′), the toilet bowl (2′28′′), Marion's dead eye (4′50′′), and the newspaper by the bed (5′24′′), all belong to the high-value group of activity signals and have stimulated higher activity signals than the other mid, panoramic, or long-range shots in the low-value group. Compared with the smaller scenes' visual simplicity and object prominence, larger scenes contain much information that distracts audiences' attention. For example, a series of shots depicting Norman walking from the hotel to the villa consists of the medium, panoramic and distant views, and the visual activity of the subjects is in a low state.
Finally, from the perspective of camera movement, the moving lens covering important narrative information triggered high-value activity in the visual brain regions. For example, at the end of the shower murder scene, the camera zooms out from a close-up of Marion's eye, pans right to the drawing, through the bathroom, and to the newspaper on the bedside table. At this point, the visual regions' activity increased from the previous low value (−0.4139) to a high value and reached a small peak (0.8155). The movement of the camera means a change in information. If the camera is the privileged perspective of the audience, then the moving camera will lead the audience to move with it, mobilize their attention, and approach or exit a scene or detail. In suspense films, the audiences' emotions are highly aroused, and at the same time, more advanced and deeper cognitive activities are also carried out in guessing and solving the mystery. Driven by this sensitivity, the moving lenses bring new clues, and its arrangement of the image is, to some extent, controlling the direction of the attention and thinking of the audience.
It is said that Hitchcock completed the principal photography of Psycho in 45 days. Of all the scenes in the film, the one on which he spent the most time was the shower murder. It took him an entire week. The scene lasted <1 min, but nearly 50 lenses were used, and the camera shifted more than 60 times. Those details of the visual design may not be recognizable, but the results have indicated that the visual brain region of the audiences has clearly and positively received and recognized the information. Therefore, although the knife never actually stabbed the actress, the film's intense visual movements created a vivid sense of violence that made audiences feel as if they were being stabbed in fear and despair as well. Regarding the visual effects of a film, Hitchcock himself once said, “The action must be divided into several details, and then jump from one detail to another. If the camera is always in the same place, acting only as of the recorder, the film cannot control the audience.” The results of this experiment also indicated that he successfully put this idea into practice. Through the editing, lens views, and camera movement, the film successfully mobilizes and affects the audience's attention and psychological state.
It is worth pointing out that although the visual design characteristics of Psycho corresponded to several visual activity signal value groups, the whole activity in the visual brain regions was still unstable and variable. Human beings mainly rely on a visual sense of living. To adapt to the complex and diverse ecological environment, the visual brain regions are extremely flexible to receive, process, and respond to numerous stimuli in daily life. They can quickly reach an excited state when facing visual stimulation. At the same time, this excited state is not easy to maintain for a long time and will rapidly decline. Hitchcock's visual design of Psycho has successfully caused several ups and downs of the visual brain activity in a short time and has seized the visual attention of the audiences. However, at the same time, its impact on the audience's cognition is not only realized by visual design. The sound design is also an important factor in making Psycho a classic, especially in the shower murder scene.
Auditory Brain Activities
The average activity in the auditory brain regions of the 10 subjects is shown in Figure 10. We corresponded the activity signal data to the film plot, and the plots which high-value groups correspond to were the shower murder (3′34′′-4′02′′), Norman's peeping (0′26′′-0′42′′), Marion tearing the paper (2′04′′-2′10′′), Marion flushing the toilet (2′20′′-2′32′′), and blood flowing into the sink or bathtub (4′42′′-5′02′′). In the climax scene or important clue scenes, the activity of auditory brain regions was activated, and the signal values were higher than average. This indicates that the audiences were led and influenced by the design of the sound in the movie.
Figure 10. (A) Average activity signals of auditory brain regions over time. (B) Time intervals with high and low auditory activity values and corresponding images.
The highest value (1.806) of auditory activity is in the classical time interval when the shrieking violin and Marion's scream almost together burst. Before that, the main sound was from Marion's relaxing shower. It was peaceful and calm. The girl finally decided to give the money back and to end all the chaos she caused. Things seemed to be getting better, and audiences also felt relaxed. The signal values at this point were in the low-value group, both in Figures 9, 10. At the moment of the mystery shadow's appearance, the sound environment suddenly switched from natural to musical, and signal values in the auditory brain region rose rapidly to the peak just in 10 s (from −0.596 to 1.806). This transition from ambient sound to the designed soundtrack is more conducive to the quick transition from a relaxed state to a highly aroused state in the auditory brain regions and also formed the climax from the auditory aspect.
Seeing the horrible shadow and the knife, Marion screamed. Her scream is a combination of fear and shock, which actually can be predicted by the audiences who have experienced all these events along with her. The actual scream was the music, the shrill sound of violins, which nobody could foresee. Its playing was so abrupt that it terrified everyone who just heard it. Cutting through the previous atmosphere of silence, it was the music that made the murder more terrifying. Bernard Herrmann, the composer of Psycho, deliberately did not add any music in the scenes just before the murder and created an atmosphere of relaxation, thus made the music more sharp and terrible. After the murder, Hitchcock chose to cut the music and slowly amplify the natural sound of running water, making the bathroom seem to be in silence. This silence beneath the natural sounds marked the end of the murder, making it full of desperation. As we can see in Figure 10, the time interval corresponding to the flowing water (4′42′′-5′02′′) is in the auditory signal's high-value group. The effect of this sound design is demonstrated by auditory brain activity.
In the history of film, Psycho is a unique case of music saving a whole movie. After seeing Psycho's rough cut, Hitchcock was so depressed that he began to think about cutting the movie and using it for television. The devastating score made him rethink this decision after Herrmann wrote the murder cue against his wishes. As he stated, “The basis of the cinema's appeal is emotional. Music's appeal is to a great extent emotional, too. To neglect music, I think, is to surrender, willfully or not, a chance to progress in filmmaking.” At the same time, the shower cue by Herrmann is also of great significance in the cultural dimension. In the 1960s, the sound of violins ripping through the silence not only brought the horror of Psycho to a climax but also implied that there was no safe harbor anymore in daily life, even if a warm and cozy bathroom was fraught with peril. It was indeed the cultural anxiety constructed through film sound.
Behavior Test Reporting Data
After the fMRI scanning, all the subjects have been asked to do the image recognition and behavior test about the memory accuracy and the emotional arousal. Following the steps stated in the method section, we have collected and calculated the reporting data. As stated in the method section, the subjects pressed button 1 if they had seen the image on the computer screen during the previous fMRI scanner and pressed button 0 if they had not. After that, they rated each image on a scale of 1–5 according to the suspense feeling they get from the image. So, in the memory accuracy test, each image had an average memory recognition score. According to the accuracy score of memory, Figure 11 has been generated. Of the screenshots of all the clips, images with the highest memory accuracy are shown in Figure 11A. Those with the lowest accuracy are shown in Figure 11B. The subjects remembered key plot points better than general plot points. For the screenshots of the scenes that included peeping, calculating the amount of money, and the appearance of the shadow, the subjects' recognition accuracy was 100%, completely consistent with the brain imaging results in the first stage of setting up the suspense, the second stage of information capture and the third stage of the climax, respectively. The close-ups of the toilet, the showerhead, the sink of the bathtub, and other images that contain important clues and are conducive to reasoning and the judgment of the plot also had 100% recognition accuracy. It indicated that the subjects formed a deeper memory of key plot points than normal or transitional plot points. From the perspective of emotion, as shown in Figure 12, the emotional rating of the subjects of the key plot points was higher than that of the general plot or transitional plot. The subjects gave high emotional scores to the suspenseful, visually stimulating close-up shots than normal transitional shots. The audience has perceived the film's audio-visual design, and the correct cognitive feedback has been formed. These results are consistent with those obtained in the fMRI experiments, which confirm that movies influence human brain cognitive activities.
Figure 11. (A) Images with 100% accuracy in the memory test. (B) Images with the lowest memory accuracy scores. Statistical significance tests have been performed on the high-value and low-value data, and p < 0.05.
Figure 12. (A) Images with high emotional rating scores. (B) Images with low emotional rating scores. Statistical significance tests have been performed on the high-value and low-value data, and p < 0.05.
However, there are also inconsistencies between the fMRI results and the results from the memory and emotion tests, which are mainly reflected in the following two aspects:
First, there was a difference in the audience's reaction to Marion's dead eye's close-up. In the memory and emotion ratings, this group of close-up shots had a high recognition accuracy and a high emotion score, but in the brain imaging data when these close-up shots appeared, the audience did not show a high intensity of brain activity, which may be different from our conclusions based on experience. It did not indicate a decline in the appeal of the film. The memory and emotion test results proved that these shots have left an impression on the audience and attracted the audience's attention. The brain-imaging data's performance might be due to the lack of cognitive engagement of the audience at the time, rather than the lack of impression or emotional enrollment. When the dangerous event in the clip was over, the audience felt relaxed and started to think about who the murderer was. Then, the audience entered into a more advanced state of thinking. In this stage, the ISC of brain activity would be reduced due to each person's different thinking modes and experience backgrounds.
Second, being the most suspenseful scene in the whole clip, the peeping scenes got high scores in brain imaging and memory accuracy. However, the emotional score was not in the high-value group, which became another contradiction.
From an experimental point of view, it could result from the participants having seen all the shots and therefore stating this was low compared to the murder shots. It could also be because the peeping scene appeared earlier in the clip when the audience felt that they had not fully entered the narration and therefore, that their emotions had not been activated. No matter what, this inconsistency, however, was what Hitchcock wanted. He talked about fear and suspense in films in an interview: “This is my specialty, I will divide it into two categories – fear and suspense. Surprise leads to fear, suspense comes from anticipation.” This suspense becomes purer and more attractive after removing the influence of emotions such as fear.
Conclusions and Outlooks
Through an fMRI scanning and behavior test, the study verified that the suspenseful movie has a stronger influence on the audience's mind than the daily non-narrative unstructured video. Additionally, this experiment has also analyzed the suspense effects of the classic shower murder in Psycho from a cognitive perspective via an empirical analysis and an audio-visual analysis. The results showed that during the process of watching the film, the average brain activity of the subjects was influenced by the content of the film, activity in the visual and auditory brain regions was also influenced by the characteristics of the visual and the soundtrack designs. The audience showed active brain cognitive activities and positive memory and emotional engagement to close-ups, quick editings, critical visual information, and dissonant music. The shower murder of Psycho is a classic that can hardly be surpassed or imitated in the history of cinema, but its success is accomplished in the most basic way: to get close and activate the audience's cognition process.
This study is a preliminary attempt to combine film study with cognitive neuroscience, especially in the aspect of the method. This study not only introduces fMRI, a technology rarely used in film studies before, but also adopts the cognitivism paradigm. Since the middle of the 1980s, cognitivism, or Post Theory, which advocated by Noël Carroll, David Bordwell, Murray Smith, Stephen Prince and others, has become a new voice in film research. They questioned the traditional film research methodology called the Grand Theory by Bordwell, and criticized its excessive structuralism tradition, which made film research driven by theories rather than specific issues. At the same time, traditional research methods are also addicted to interpreting the film content, avoiding reasoning, and logical argumentation. Most of all, the audience's psychological activities are classified into the unconscious and lose the initiative. Stephen Prince used the phrase “the missing spectator” to refer to the absence of the audience's perspective in film study. He believed that psychoanalysis's traditional film theory ignored the important role of the perceptual process, which is one of the central issues concerned by the cognitive theory. Noël Carroll and David Bordwell put forward “piecemeal theory” and “middle-level issue,” respectively, emphasizing that film research should focus on specific issues and carry out more research and theoretical exploration, which are performed bottom-up. Although cognitivism does not emphasize that it must be studied empirically, this study is a practice and enrichment of cognitivism. It has been a cliche that Psycho is engrossing and owns millions of fans. Rather than analyzing the techniques used in the film or reading the creators' interviews repeatedly, it is better to switch perspectives and methods, to explore what specific cognitive processes and emotional experiences the audience had while watching this film.
Regarding specific methods and operations, simultaneous cognitive activities were obtained by fMRI scanning during the watching process. Compared with the traditional effect research based on questionnaires and interviews, the results of brain activity signals were not obtained after the movie watching but were obtained simultaneously with the watching behavior. This method of film effects measuring has the potential to be used in more future areas such as pre-testing of commercial films or film classification. Through this new approach, deeper and more precise answers from the audience will come to the stage.
It should be noted that in this experiment, the division of subsections of movie clips is not accurate. Due to film information transmission and reception's complexity, the audience's response presents a continuous state. We established the starting and ending points of each section according to the changes in the data. However, in the human brain's complex cognitive process, strict starting and ending points do not truly exist. In future studies, we will adopt behavioral experiments, eye movement measurements, and other means to improve these deficiencies.
This study was carried out in a laboratory environment that was independent, closed, and non-social. Movies are often simultaneously watched by crowds. There will inevitably be some contradictions between the two environments. Limited by the existing experimental conditions, it is difficult to perform undetectable collective brain scanning at present. Research on the combination of film and neuroscience still has a long way to go.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics Statement
The studies involving human participants were reviewed and approved by Institutional Review Board of the State Key Laboratory of Cognitive Neuroscience and Learning at Beijing Normal University. The patients/participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.
Author Contributions
YaW was responsible for data processing and article writing, while YiW was responsible for experimental design, article modification, and proofreading. All authors contributed to the article and approved the submitted version.
Funding
This study was financially sponsored through Beijing Municipal Philosophy and Social Science Planning Office (Grant No. 16YTA002) and National Social Science Fund of Art (Grant No. 19BC041). The authors thank them for financial support.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
We would like to thank Professor Yong He and Dr. Jin Liu of IDG/Mc GOVERN INSTITUE—BNU for their help in data collection processing and analysis. As an interdisciplinary research study, this paper benefits a lot from IDG/Mc GOVERN INSTITUE—BNU. The authors would like to thank the following people for their assistance in the early stage, including materials preparing, data collecting, and participants recruitment. They are Zhichen Shi, Quanquan Wang, Wanyi Zhang, and Xiaoyang Li.
References
Banash, D. (2015). Alfred Hitchcock's “Psycho” and the cinematic novels of don delillo and manuel muñoz. Literat. Film Q 43, 4–17. Available online at: https://www.jstor.org/stable/ 43799006
Belton, J. (2003). Can Hitchcock be saved from hitchcock studies? Cinéaste 28, 16–21. Available online at: https://www.jstor.org/stable/41689632
Bondebjerg, I. (2014). Documentary and cognitive theory: narrative, emotion and memory. Media Commun. 2,13–22. doi: 10.17645/mac.v2i1.17
Bondebjerg, I. (2017). The creative mind: cognition, society and culture. Palgrave Commun. 3, 1–7. doi: 10.1057/s41599-017-0024-1
Bordwell, D. (2010). The part-time cognitivist: a view from film studies. Projections 4, 1–18. doi: 10.3167/proj.2010.040202
Cheong, Y. G., and Young, R. M. (2008). Narrative generation for suspense: modeling and evaluation. U. Spierling N. Szilas 5334, 144–155. doi: 10.1007/978-3-540-89454-4_21
Chestnutt, I. G., Morgan, M. Z., Hoddell, C., and Playle, R. (2004). A Comparison of a computer-based questionnaire and personal interviews in determining oral health-related behaviours. Commun. Dent Oral Epidemiol. 32, 410–417. doi: 10.1111/j.1600-0528.2004.00160.x
Comisky, P., and Bryant, J. (1982). Factors involved in generating suspense. Hum. Commun. Res. 9, 49–58. doi: 10.1111/j.1468-2958.1982.tb00682.x
Cutting, J. E., DeLong, J. E., and Brunick, K. L. (2011). Visual activity in hollywood film: 1935 to 2005 and beyond. Psychol. Aesthetics Creat. Arts 5, 115–125. doi: 10.1037/a0020995
Dmochowski, J. P., Bezdek, M. A., Abelson, B. P., Johnson, J. S., Schumacher, E. H., and Parra, L. C. (2014). Audience preferences are predicted by temporal reliability of neural processing. Nat. Commun. 5:4567. doi: 10.1038/ncomms5567
Dudai, Y. (2012). The cinema-cognition dialogue: a match made in brain. Front. Hum. Neurosci. 1:248. doi: 10.3389/fnhum.2012.00248
Francuz, P., and Mendyk, E. Z. (2014). Does the brain differentiate between related and unrelated cuts when processing audiovisual messages? An ERP study. Media Psychol. 16, 461–475, doi: 10.1080/15213269.2013.831394
Gerrig, R. J. (1993). Experiencing Narrative Worlds: On the Psychological Activities of Reading. New Haven, CT, Yale University Press.
Grill-Spector, K., and Sayres, R. (2008). Object recognition: insights from advances in fMRI methods. Curr. Direct. Psychol. Sci. 17, 73–79. doi: 10.1111/j.1467-8721.2008.00552.x
Gross, C., and De Schonen, S. (1992). Representation of visual stimuli in inferior temporal cortex [and discussion]. Philos. Trans. Biol. Sci. 335, 3–10. doi: 10.1098/rstb.1992.0001
Gruskin, D. C., Rosenberg, M. D., and Holmes, A. J. (2019). Relationships between depressive symptoms and brain responses during emotional movie viewing emerge in adolescence. Neuroimage 216, 1–12. doi: 10.1016/j.neuroimage.2019.116217
Hasson, U., Furman, O., Clark, D., Dudai, Y., and Davachi, L. (2008a). Enhanced intersubject correlations during movie viewing correlate with successful episodic encoding. Neuron 57, 452–462. doi: 10.1016/j.neuron.2007.12.009
Hasson, U., Landesman, O., Knappmeyer, B., Vallines, I., Rubin, N., and Heeger, D. J. (2008b). Neurocinematics: the neuroscience of film. Projections 2, 1–26. doi: 10.3167/proj.2008.020102
Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., and Malach, R. (2004). Intersubject synchronion of cortical activity during natural vision. Science 303, 1634–1640. doi: 10.1126/science.1089506
Hinkin, T. R. (1995). A review of scale development in the study of behavior in organizations. J. Manag. 21, 967–988. doi: 10.1177/014920639502100509
Kwak, J., and Zhang, L. (2011). Does china love hollywood? An empirical study on the determinants of the box-office performance of the foreign films in China. Int. Area Stud. Rev. 14, 116–136. doi: 10.1177/223386591101400205
Lahnakoski, J. M., Jääskeläinen, I. P., Sams, M., and Nummenmaa, L. (2017). Neural mechanisms for integrating consecutive and interleaved natural events. Hum. Brain Map. 38, 3360–3376. doi: 10.1002/hbm.23591
Lehne, M., and Koelsch, S. (2015). Toward a general psychological model of tension and suspense. Hypothesis Theory Article 6, 1–11. doi: 10.3389/fpsyg.2015.00079
Logothetis, N. K., Pauls, J., Auguth, M., Trinath, T., and Oeltermann, A. (2001). A neurophysiological investigation of the basis of the BOLD signal in fMRI. Nature 412, 150–157. doi: 10.1038/35084005
Motz, B. (2013). Cognitive science in popular film: the cognitive science movie index. Trends Cogn. Sci. 17, 483–485. doi: 10.1016/j.tics.2013.08.002
Naab, T., and Sukalla, F. (2019). Hero or Villain? the role of audience beliefs about suspense for their suspense experience. Stud. Commun. Media 8, 53–76. doi: 10.5771/2192-4007-2019-1-53
Owen, A., Stern, C., Look, R., Tracey, I., Rosen, B., and Petrides, M. (1998). Functional organization of spatial and nonspatial working memory processing within the human lateral frontal cortex. Proc. Natl. Acad. Sci. U.S.A. 95, 7721–7726. doi: 10.1073/pnas.95.13.7721
Pehrs, C., Deserno, L., Bakels, J. H., Schlochtermeier, L. H., Kappelhoff, H., Jacobs, A. M., et al. (2014). How music alters a kiss: superior temporal gyrus controls fusiform–amygdala effective connectivity. Soc. Cogn. Affect. Neurosci. 9, 1770–1778. doi: 10.1093/scan/nst169
Plantinga, C. (2012). Art moods and human moods in narrative cinema. New Literary Hist. 43, 455–475. doi: 10.1353/nlh.2012.0025
Raz, G., and Hendler, T. (2014). Forking cinematic paths to the self: neurocinematically informed model of empathy in motion pictures. Projections 8, 89–114. doi: 10.3167/proj.2014.080206
Rebello, S. (2004). From Alfred Hitchcock and the Making of Psycho. Alfred Hitchcock's Psycho: A Casebook. Transl. by R. Kolker. New York, NY: Oxford University Press, 52.
Rooney, B., and Hennessy, E. (2013). Actually in the cinema: a field study comparing real 3D and 2D movie patrons' attention, emotion, and film satisfaction. Media Psychol. 16, 441–460. doi: 10.1080/15213269.2013.838905
Stadler, J. (2018). “Mind the gap”: between movies and mind, affective neuroscience, and the philosophy of film. Projections 12, 86–94. doi: 10.3167/proj.2018.120211
Tal-Or, N, and Cohen, J. (2010). Understanding audience involvement: conceptualizing and manipulating identification and transportation. Poetics 38, 402–418. doi: 10.1016/j.poetic.2010.05.004
Thomson, D. (2009). The Moment of Psycho: How Alfred Hitchcock Taught America to Love Murder. New York, NY: Basic Books.
Willems, R. M., Clevis, K., and Hagoort, P. (2011). Add a picture for suspense: neural correlates of the interaction between language and visual information in the perception of fear. Soc. Cogn. Affect. Neurosci. 6, 404–416. doi: 10.1093/scan/nsq050
Keywords: neurocinematics, fMRI, suspense, Hitchcock, brain
Citation: Wang Y and Wang Y (2020) A Neurocinematic Study of the Suspense Effects in Hitchcock's Psycho. Front. Commun. 5:576840. doi: 10.3389/fcomm.2020.576840
Received: 27 June 2020; Accepted: 26 October 2020;
Published: 12 November 2020.
Edited by:
Douglas Ashwell, Massey University Business School, New ZealandReviewed by:
Paromita Pain, University of Nevada, Reno, United StatesNiki Murray, Massey University Business School, New Zealand
Copyright © 2020 Wang and Wang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Yiwen Wang, wyiwbnu@126.com