Marcus Rothkirch
Guido Hesselmann- Visual Perception Laboratory, Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité – Universitätsmedizin Berlin, Berlin, Germany
In this perspective article, we first outline the large diversity of methods, measures, statistical analyses, and concepts in the field of the experimental study of unconscious processing. We then suggest that this diversity implies that comparisons between different studies on unconscious processing are fairly limited, especially when stimulus awareness has been assessed in different ways. Furthermore, we argue that flexible choices of methods and measures will inevitably lead to an overestimation of unconscious processes. In the concluding paragraph, we briefly present solutions and strategies for future research. We make a plea for the introduction of “best practices,” similar to previous attempts to constitute practicing standards for functional magnetic resonance imaging (fMRI) and electroencephalography (EEG).
Introduction
To what extent are our actions and thoughts determined by “unconscious” influences, that is, by influences that we ourselves are not aware of? Sixty years after the infamous marketing campaign launched by James Vicary in 1957, who claimed that unconsciously flashed messages can affect observers’ consumption behavior (Karremans et al., 2006), this question has lost nothing of its fascination. What is more, the development of scientific methods, like neuroscientific instruments or psychophysical “blinding” techniques, has made it possible to assess unconscious processes from different perspectives and within a wide range of contexts. There is, however, a downside to this multifaceted approach to unconscious processing: It comes along with substantial diversity between scientific studies.
In the following sections, we will first provide a brief overview of aspects in consciousness research that are affected by this diversity. Afterward, we will outline the risks associated with such divergent approaches. We will conclude with a section on practical recommendations for future research.
Diversities Regarding Different Experimental Aspects
Diversity in Suppression Techniques
Today, a wide range of psychophysical paradigms exist to experimentally manipulate sensory awareness, particularly within the visual domain (Bachmann et al., 2007). Such psychophysical “blinding” methods can evoke a transient blindness in neurologically intact observers, i.e., they can be used to render a physically presented target stimulus (or, features of it) invisible, in most cases for a limited amount of time. It is important to keep in mind that the available paradigms vary with respect to what types of stimuli can be suppressed from awareness, and how effective the suppression is, for instance in terms of duration and predictability of suppression onset and offset (Kim and Blake, 2005). But there is yet another level of diversity related to the “blinding” methods that has only recently begun to be studied systematically. In 2014, a Frontiers Research Topic [“Invisible, but how?”; edited by (Dubois and Faivre, 2014)] addressed the issue of the inherent differences in the amount of information let through by different suppression techniques. Put simply, it is not enough to know that a stimulus has been invisible, but it is crucial to also know how exactly invisibility was achieved, because it has become apparent that different levels of suppression depth are associated with different suppression methods. Recently, this notion has been formalized into a functional hierarchy of unconscious visual processing (Breitmeyer, 2014). Eventually, such a functional hierarchy will allow to formulate predictions about the level of unconscious processing that can be expected in a specific experimental setup (Breitmeyer, 2015).
In the absence of such prior assumptions on the depth of visual suppression associated with a specific paradigm, every new study reporting high-level unconscious processing is equally weighted, independently of the applied suppression method. When accumulating evidence across single studies, this might lead to the wrong conclusion that unconscious perception is unlimited. However, it is known that different suppression methods are associated with different levels of suppression. Therefore, for example, the evidence presented for unconscious processing under CFS should be stronger, as a plausible prior assumption could be that CFS leads to a relatively high level of suppression. Ideally, the functional hierarchy would be based on cumulative research strategies, and incorporate not only studies that directly compared results from different methods with each other (Izatt et al., 2014; Peremen and Lamy, 2014a), but also take data from neuroimaging studies into account (Fogelson et al., 2014; Ludwig et al., 2016).
Diversity in Awareness Measures
The question of how to optimally measure awareness in experiments on unconscious processing has been a long-standing one, and the issue is still much debated (e.g., Merikle and Reingold, 1990; Kunimoto et al., 2001; Schmidt and Vorberg, 2006; Sandberg et al., 2010). While criteria for valid awareness measures have been formulated (Shanks and John, 1994; De Houwer et al., 2009; Newell and Shanks, 2014), there is no accepted “gold standard.” We will briefly outline the diversity of frequently applied measures of awareness, but this overview is not meant to be exhaustive. One crucial differentiation is the distinction between objective and subjective measures of awareness. According to the objective awareness criterion, forced-choice detection or discrimination performance above chance level indicates stimulus awareness, while performance at chance level indicates its absence. Subjective measures of awareness, on the other hand, are based on participants’ metacognitive judgements on their own mental states (Lau and Rosenthal, 2011). Usually participants perform such judgments either on their own experience of the stimulus, or on their accuracy in a discrimination task. In the first case, participants are required to rate the visibility of the stimulus, either on a larger (Sergent and Dehaene, 2004) or smaller scale (Ramsøy and Overgaard, 2004). In the second case, participants have to evaluate how confident they felt with their response in a previously performed discrimination task (e.g., Rothkirch et al., 2012). The pros and cons of objective and subjective awareness measures have been summarized elsewhere (e.g., Hesselmann, 2013). A further distinction can be made between filtering measures and aggregate measures of awareness. If behavioral reports are provided on a trial-by-trial basis during the main experiment (“online”), filtering allows the experimenter to post-select subsets of trials for further analysis (e.g., “seen” vs. “not seen”). By contrast, aggregate measures allow inferences only about blocks of trials and thus depend on more than a single trial (e.g., percent correct, d’). Behavioral reports may be recorded either after a block of trials or in a separate control experiment (“offline,” or “two-task design”). If a pre-defined criterion, such as performance at chance level, is satisfied, the experimenter infers that stimulus awareness was absent during the main experiment (or, in a given block of trials). Importantly, diversity does not end here, and the devil is in the details. For example, it has been shown that the exact composition of a block of trials (e.g., blocks with weakly and fully visible stimuli, vs. blocks with only weakly visible stimuli) can influence the experienced level of awareness [(Lin and Murray, 2014); also see (Pratte and Rouder, 2009)]. Levels of stimulus awareness may also change in longer experiments, e.g., awareness can increase across trials due to perceptual learning (Schwiedrzik et al., 2011; Ludwig et al., 2013). Finally, the presence or absence of trial-by-trial awareness reports can in turn influence the effect of interest, e.g., response priming (Peremen and Lamy, 2014b).
Diversity in Statistical Analysis
We will briefly illustrate the diversity in the statistical analysis of awareness test data by an example. It is not uncommon that participants perform a two-alternative forced-choice (2AFC) discrimination task in a control experiment, either in addition to a subjective awareness measure (e.g., Hesselmann et al., 2016), or without any further awareness tests during the main experiment (e.g., Koechlin et al., 1999; Mattler and Palmer, 2012). To show that participants did not perceive the presented stimuli, researchers then often apply null hypothesis significance testing (NHST) on the 2AFC data, either at the subject level or at the group level. Alternatively, the researcher may choose to apply a cutoff, e.g., consider 2AFC performance below 60% as indicative of an absence of stimulus awareness. It is well understood that when the NHST procedure is underpowered (or, the power is unknown), the experimenter risks to falsely accept the null hypothesis, i.e., conclude that there is no deviation from chance level when there actually is a deviation (Vadillo et al., 2016b). For example, the number of trials included in the awareness test may determine whether participants are categorized as aware or unaware, simply because tests comprising more trials also yield more sensitive measures. There have been suggestions to avoid the problem of type II errors by using equivalence tests and equivalence confidence intervals (Overgaard et al., 2013). A challenge related to such equivalence tests, however, is the a priori definition of a boundary around chance level that is still considered acceptable to indicate unawareness (Lin and Murray, 2015). Recently, Bayesian statistics have more frequently been applied to establish chance performance (e.g., Dienes, 2015; Sand and Nilsson, 2016). The details of this approach are beyond the scope of this perspective article, but it is important to keep in mind that Bayesian statistics are inherently diverse too. For example, when using Bayes factors to estimate the evidence in favor of the null model, there are different options for specifying the predictions of the alternative model (i.e., the case where participants saw the stimulus). One particularly relevant feature of the Bayesian approach seems to be sequential sampling, i.e., the strategy to collect more data until the evidence in favor of one model is considered as conclusive. Beyond the question of NHST or Bayesian statistics, post hoc selection of data has frequently been used, for example when residual stimulus visibility greatly varies between participants in experiments using interocular suppression (e.g., Sklar et al., 2012). The idea is to perform the main statistical analysis on a post hoc selection of the recorded data, e.g., by excluding participants whose subjective or objective behavioral reports indicated stimulus awareness, or by exclusively analyzing “not seen” trials. It has, however, been criticized to interpret “not seen” trials purely in isolation (Schmidt, 2015). Instead, these trials should be contrasted against seen trials, i.e., trials in which the suppression of the critical stimulus was not successful (cf. Madipakkam et al., 2015). Furthermore, especially in cases in which a large number of trials or participants are excluded, data analysis can resemble some form of extreme group analysis. Due to regression to the mean such an analysis strategy can yield a statistical bias in the selected sample and let the researcher erroneously assume unawareness in the selected participants (Shanks, 2016). The magnitude of regression to the mean in a data set is indicated by the correlation between the awareness measure and the measured effect. In this context, a model taking only into account regression to the mean can function as a null model against which the experimental effect can be tested. It has to be noted, however, that the extent to which the experimental effect differs from such a null model neither conclusively demonstrates nor precludes an unconscious effect.
Diversity in Experimental Setting
In a typical psychological experiment, stimuli are presented to participants who are asked to elicit a particular response to these stimuli. If such an experiment is intended to investigate unconscious processes, the unconscious dimension could be related to different aspects of the experimental setup. Firstly, participants could be unconscious of the presented stimulus, which is usually achieved by some form of masking (Kim and Blake, 2005). The second category is constituted by studies in which the putatively unconscious process pertains to the relationship between the stimulus and participants’ behavior. This is, for instance, typically the case in studies on social priming (Bargh, 2016). Finally, the behavioral response or sequence of responses elicited by the participants might be unconscious, as in studies focusing on learning processes (Destrebecqz and Cleeremans, 2001).
While in cognitive psychology the focus lies primarily on the unawareness of the stimuli, in social psychology the relation between stimuli and observers’ behavior is often the primary target (Doyen et al., 2014). Especially in the latter case, however, it proves difficult to determine whether the process of interest is indeed unconscious. As pointed out by (Stafford, 2014), for instance, the degree to which observers lack knowledge about associations between stimuli and their behavior that is required to be considered ‘unconscious’ is conceptually elusive. An ostensible remedy often chosen in the face of this predicament is then to mask the stimulus such that it cannot be consciously perceived anymore. This demonstrates that the different approaches to study unconscious processes are often used as if they were interchangeable, although they might target distinct processes.
Remarkably, it is widely accepted, on the one hand, that consciousness is not a unitary process, but that there are instead distinct modes of ‘consciousness,’ which, for instance, implies that observers might have conscious knowledge about one aspect of a particular stimulus while lacking conscious access to other aspects of that same stimulus (Zeki and Bartels, 1998; Navajas et al., 2014). In contrast, however, unconsciousness is sometimes treated as a singular process that does not require a further differentiation, although the term “unconscious” is not unitary either (Moors and De Houwer, 2006).
Consequences of the Outlined Diversities
As summarized above, researchers have several options to study and identify unconscious processes, ranging from the experimental design to the statistical analysis of the acquired data. While this patchwork of approaches could be deemed beneficial in the sense that it entails a more extensive overview of the phenomenon of unconscious processing, we argue that it more likely has the opposite effect. It should be noted that the diversities detailed above have distinct implications. On the one hand, they can have direct consequences on the interpretability of findings from single studies, which can, in severe cases, imply that participants were actually not unaware of the presumed unconscious process. Such consequences are especially related to the diversities discussed in section “Diversity in Awareness Measures and Diversity in Statistical Analysis.” On the other hand, the diversities regarding the experimental setting (see section Diversity in Experimental Setting) and the suppression techniques (see section Diversity in Suppression Techniques) have less impact on single studies, but play a more important role at the conceptual level, that is, when the findings across several studies are intended to be integrated into a general framework or model of (un)consciousness.
Specifically, studies differ with respect to the way awareness or unawareness, respectively, is assessed and statistically analyzed. This means that different criteria are set to determine whether participants are aware or unaware of a particular stimulus or process. For a given study, it is thus not sufficient to simply know whether participants were unaware, but instead according to which criterion participants were unaware, since it cannot be precluded that other criteria might indicate participants’ awareness of the critical stimulus or process. This, however, limits the comparability between different studies, especially when awareness has been assessed in different ways. For example, two functional magnetic resonance imaging (fMRI) studies have investigated neural activity in visual areas when images of objects were rendered invisible with CFS (Fang and He, 2005; Hesselmann and Malach, 2011). In one study (Fang and He, 2005), participants were asked to report whether they perceived any shape or object in the preceding block of trials. In the other case (Hesselmann and Malach, 2011), participants provided trial-by-trial visibility ratings on a 3-point scale. It could be argued that only a limited comparison of the effect of interest is possible between the two studies, since evidence for participants’ unawareness was based on different types of responses.
Another critical aspect that especially pertains to the assessment of observers’ awareness is the flexibility in data analysis based on the vast amount of awareness definitions. Simply put, if researchers are interested in demonstrating that observers were unaware they could choose to report a particular analysis that indeed speaks in favor of observers’ unawareness, while neglecting that other criteria might have indicated awareness. For example, when faced with the divergence of subjective and objectives measures in some participants, researchers may argue in terms of “blindsight” in normal observers (Hesselmann et al., 2011), criticize the underlying assumption of different objective and subjective thresholds (Peters and Lau, 2015), or choose one or the other measure to demonstrate observers’ unawareness. Such a flexible choice could overstretch the concept of unawareness and consequently lead to an overestimation of unconscious processes, as the researcher has the “freedom” to report only the particular analysis that confirms their expectation (cf. Simmons et al., 2011).
On a more conceptual level, the vagueness of the terms “awareness” and “consciousness” allows one to operate with them in different contexts (Moors and De Houwer, 2006). If two different studies make reference to “unconscious processes,” the reader may likely assume that similar processes have been studied, although one study may, say, use visually masked stimuli while the other one may focus on “unconscious processes” in response to visible stimuli. Such an inflationary reference to “awareness” and “consciousness” bears the risk of counteracting a differentiation between qualitatively distinct phenomena and may eventually – perhaps even inevitably – lead to the claim that unconscious processes can carry out every fundamental high-level cognitive function that conscious processes can perform [(Hassin, 2013); for a reply (Hesselmann and Moors, 2015)], and thus impede deeper insights into the nature of unconscious processes.
Finally, one can speculate that the outlined diversities also contribute to the current crisis of confidence in psychological research. Some of the findings on unconscious processes, including, among others, effects of social priming and unconscious perception, turned out to be difficult to replicate (Doyen et al., 2012; Hesselmann and Knops, 2014; Moors et al., 2016), show clear signs of bias (Shanks et al., 2015; Vadillo et al., 2016a), or are open to alternative explanations (Stein et al., 2016; Street and Vadillo, 2016).
Solutions and Strategies for Future Research
One obvious strategy to handle the described diversity is the formulation of explicit guidelines for the assessment and statistical analysis of awareness. Such guidelines may not only help to unify the fragmented approaches that exist to study unconscious processing, but also may serve as a guide for researchers who are not yet acquainted with the peculiarities and pitfalls related to the assessment of observers’ level of awareness. They should, however, not be understood as a discrete set of binding rules. Instead, we propose that the whole field should strive for the establishment of ‘best practices,’ similar to previous attempts to constitute practicing standards for fMRI (Poldrack et al., 2017) and EEG research (Picton et al., 2000). For example, the “21-word-solution” (Simmons et al., 2012) that is intended to make the post hoc exclusion of conditions and/or participants transparent, could easily be adopted by the field, given the fact that post hoc exclusions may indeed cause serious problems in studies on unconscious processing (Shanks, 2016). Ideally, full transparency of methods should be the norm. Notably, the current diversity between studies will not be completely abolished by such an intervention, but only reduced. The definition of an overarching ‘gold standard’ would not be feasible, since, for instance, the possibilities to assess awareness are often limited by the particular experimental design.
Moreover, it should be acknowledged that qualitatively distinct phenomena are subsumed under the umbrella terms “consciousness” and “unconsciousness.” To increase the understanding of the scope and limits of unconscious processes, however, such a deficient differentiation seems counterproductive. Thus, to reflect and accentuate that research on unconscious processing stems from a variety of experimental settings, a more fine-grained taxonomy of unconscious phenomena seems expedient. For instance, studies in which the processing of visually masked stimuli is investigated could consistently label “unconscious processing” as “subliminal processing” (Dehaene et al., 2006). In contrast, if supraliminal information is processed unbeknownst to the observer in an incidental manner, as in the case of “unconscious learning processes,” one could label this as “implicit processing.” The choice of such terms should also be tailored to their underlying assumptions, as in the case of “subliminal” for instance, which could suggest a high-threshold model. Thus, other labels may be more suitable. Especially to keep up with the development of new techniques, methods, and experimental designs, however, such a differentiation is generally advisable.
Finally, we believe that a great service to this field will be done by an adherence to openly available data, so that the results can be reproduced, and the conclusions tested against alternative statistical analyses.
Author Contributions
MR and GH: Conception of the manuscript, Preparation of a first draft, Contribution of critical input and finalization of the manuscript.
Funding
MR and GH were supported by the German Research Foundation (grants RO 4836/2-1 & HE 6244/1-2).
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Bachmann, T., Breitmeyer, B., and Ogmen, H. (2007). The Experimental Phenomena of Consciousness: A Brief Dictionary. Oxford: Oxford University Press.
Bargh, J. A. (2016). Awareness of the prime versus awareness of its influence: implications for the real-world scope of unconscious higher mental processes. Curr. Opin. Psychol. 12, 49–52. doi: 10.1016/j.copsyc.2016.05.006
Breitmeyer, B. (2014). “Functional hierarchy of unconscious object processing,” in The Visual (Un)Conscious and Its (Dis)Contents: A microtemporal approach, (Oxford: Oxford University Press), 89–102.
Breitmeyer, B. G. (2015). Psychophysical “blinding” methods reveal a functional hierarchy of unconscious visual processing. Conscious. Cogn. 35, 234–250. doi: 10.1016/j.concog.2015.01.012
De Houwer, J., Teige-Mocigemba, S., Spruyt, A., and Moors, A. (2009). Implicit measures: a normative analysis and review. Psychol. Bull. 135, 347–368. doi: 10.1037/a0014211
Dehaene, S., Changeux, J.-P., Naccache, L., Sackur, J., and Sergent, C. (2006). Conscious, preconscious, and subliminal processing: a testable taxonomy. Trends Cogn. Sci. 10, 204–211. doi: 10.1016/j.tics.2006.03.007
Destrebecqz, A., and Cleeremans, A. (2001). Can sequence learning be implicit? New evidence with the process dissociation procedure. Psychon. Bull. Rev. 8, 343–350.
Dienes, Z. (2015). “How bayesian statistics are needed to determine whether mental states are unconscious,” in Behavioural Methods in Consciousness Research, ed. M. Overgaard (Oxford: Oxford University Press), 199–220.
Doyen, S., Klein, O., Pichon, C.-L., and Cleeremans, A. (2012). Behavioral priming: it’s all in the mind, but whose mind? PLoS ONE 7:e29081. doi: 10.1371/journal.pone.0029081
Doyen, S., Klein, O., Simons, D. J., and Cleeremans, A. (2014). On the other side of the mirror: priming in cognitive and social psychology. Soc. Cogn. 32, 12–32. doi: 10.1521/soco.2014.32.supp.12
Dubois, J., and Faivre, N. (2014). Invisible, but how? The depth of unconscious processing as inferred from different suppression techniques. Front. Psychol. 5:1117. doi: 10.3389/fpsyg.2014.01117
Fang, F., and He, S. (2005). Cortical responses to invisible objects in the human dorsal and ventral pathways. Nat. Neurosci. 8, 1380–1385. doi: 10.1038/nn1537
Fogelson, S. V., Kohler, P. J., Miller, K. J., Granger, R., and Tse, P. U. (2014). Unconscious neural processing differs with method used to render stimuli invisible. Front. Psychol. 5:601. doi: 10.3389/fpsyg.2014.00601
Hassin, R. R. (2013). Yes it can: on the functional abilities of the human unconscious. Perspect. Psychol. Sci. 8, 195–207. doi: 10.1177/1745691612460684
Hesselmann, G. (2013). Dissecting visual awareness with FMRI. Neuroscientist 19, 495–508. doi: 10.1177/1073858413485988
Hesselmann, G., Darcy, N., Ludwig, K., and Sterzer, P. (2016). Priming in a shape task but not in a category task under continuous flash suppression. J. Vis. 16, 17. doi: 10.1167/16.3.17
Hesselmann, G., Hebart, M., and Malach, R. (2011). Differential BOLD activity associated with subjective and objective reports during “blindsight” in normal observers. J. Neurosci. 31, 12936–12944. doi: 10.1523/JNEUROSCI.1556-11.2011
Hesselmann, G., and Knops, A. (2014). No conclusive evidence for numerical priming under interocular suppression. Psychol. Sci. 25, 2116–2119. doi: 10.1177/0956797614548876
Hesselmann, G., and Malach, R. (2011). The link between fMRI-BOLD activation and perceptual awareness is “stream-invariant” in the human visual system. Cereb. Cortex 21, 2829–2837. doi: 10.1093/cercor/bhr085
Hesselmann, G., and Moors, P. (2015). Definitely maybe: can unconscious processes perform the same functions as conscious processes? Front. Psychol. 6:584. doi: 10.3389/fpsyg.2015.00584
Izatt, G., Dubois, J., Faivre, N., and Koch, C. (2014). A direct comparison of unconscious face processing under masking and interocular suppression. Front. Psychol. 5:659. doi: 10.3389/fpsyg.2014.00659
Karremans, J. C., Stroebe, W., and Claus, J. (2006). Beyond vicary’s fantasies: the impact of subliminal priming and brand choice. J. Exp. Soc. Psychol. 42, 792–798. doi: 10.1016/j.jesp.2005.12.002
Kim, C.-Y., and Blake, R. (2005). Psychophysical magic: rendering the visible “invisible.” Trends Cogn. Sci. 9, 381–388. doi: 10.1016/j.tics.2005.06.012
Koechlin, E., Naccache, L., Block, E., and Dehaene, S. (1999). Primed numbers: exploring the modularity of numerical representations with masked and unmasked semantic priming. J. Exp. Psychol. Hum. Percept. Perform. 25, 1882–1905. doi: 10.1037/0096-1523.25.6.1882
Kunimoto, C., Miller, J., and Pashler, H. (2001). Confidence and accuracy of near-threshold discrimination responses. Conscious. Cogn. 10, 294–340. doi: 10.1006/ccog.2000.0494
Lau, H., and Rosenthal, D. (2011). Empirical support for higher-order theories of conscious awareness. Trends Cogn. Sci. 15, 365–373. doi: 10.1016/j.tics.2011.05.009
Lin, Z., and Murray, S. O. (2014). Priming of awareness or how not to measure visual awareness. J. Vis. 14, 27. doi: 10.1167/14.1.27
Lin, Z., and Murray, S. O. (2015). Automaticity of unconscious response inhibition: comment on Chiu and Aron (2014). J. Exp. Psychol. Gen. 144, 244–254. doi: 10.1037/xge0000042
Ludwig, K., Sterzer, P., Kathmann, N., Franz, V. H., and Hesselmann, G. (2013). Learning to detect but not to grasp suppressed visual stimuli. Neuropsychologia 51, 2930–2938. doi: 10.1016/j.neuropsychologia.2013.09.035
Ludwig, K., Sterzer, P., Kathmann, N., and Hesselmann, G. (2016). Differential modulation of visual object processing in dorsal and ventral stream by stimulus visibility. Cortex 83, 113–123. doi: 10.1016/j.cortex.2016.07.002
Madipakkam, A. R., Rothkirch, M., Guggenmos, M., Heinz, A., and Sterzer, P. (2015). Gaze direction modulates the relation between neural responses to faces and visual awareness. J. Neurosci. 35, 13287–13299. doi: 10.1523/JNEUROSCI.0815-15.2015
Mattler, U., and Palmer, S. (2012). Time course of free-choice priming effects explained by a simple accumulator model. Cognition 123, 347–360. doi: 10.1016/j.cognition.2012.03.002
Merikle, P. M., and Reingold, E. M. (1990). Recognition and lexical decision without detection: unconscious perception? J. Exp. Psychol. Hum. Percept. Perform. 16, 574–583.
Moors, A., and De Houwer, J. (2006). Automaticity: a theoretical and conceptual analysis. Psychol. Bull. 132, 297–326. doi: 10.1037/0033-2909.132.2.297
Moors, P., Boelens, D., van Overwalle, J., and Wagemans, J. (2016). Scene integration without awareness: no conclusive evidence for processing scene congruency during continuous flash suppression. Psychol. Sci. 27, 945–956. doi: 10.1177/0956797616642525
Navajas, J., Rey, H. G., and Quian Quiroga, R. (2014). Perceptual and contextual awareness: methodological considerations in the search for the neural correlates of consciousness. Front. Psychol. 5:959. doi: 10.3389/fpsyg.2014.00959
Newell, B. R., and Shanks, D. R. (2014). Unconscious influences on decision making: a critical review. Behav. Brain Sci. 37, 1–19. doi: 10.1017/S0140525X12003214
Overgaard, M., Lindeløv, J., Svejstrup, S., Døssing, M., Hvid, T., Kauffmann, O., et al. (2013). Is conscious stimulus identification dependent on knowledge of the perceptual modality? Testing the “source misidentification hypothesis.” Front. Psychol. 4:116. doi: 10.3389/fpsyg.2013.00116
Peremen, Z., and Lamy, D. (2014a). Comparing unconscious processing during continuous flash suppression and meta-contrast masking just under the limen of consciousness. Front. Psychol. 5:969. doi: 10.3389/fpsyg.2014.00969
Peremen, Z., and Lamy, D. (2014b). Do conscious perception and unconscious processing rely on independent mechanisms? A meta-contrast study. Conscious. Cogn. 24, 22–32. doi: 10.1016/j.concog.2013.12.006
Peters, M. A. K., and Lau, H. (2015). Human observers have optimal introspective access to perceptual processes even for visually masked stimuli. eLife 4:e09651. doi: 10.7554/eLife.09651
Picton, T. W., Bentin, S., Berg, P., Donchin, E., Hillyard, S. A., Johnson, R., et al. (2000). Guidelines for using human event-related potentials to study cognition: recording standards and publication criteria. Psychophysiology 37, 127–152.
Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., et al. (2017). Scanning the horizon: towards transparent and reproducible neuroimaging research. Nat. Rev. Neurosci. 18, 115–126. doi: 10.1038/nrn.2016.167
Pratte, M. S., and Rouder, J. N. (2009). A task-difficulty artifact in subliminal priming. Atten. Percept. Psychophys. 71, 1276–1283. doi: 10.3758/APP.71.6.1276
Ramsøy, T. Z., and Overgaard, M. (2004). Introspection and subliminal perception. Phenomenol. Cogn. Sci. 3, 1–23. doi: 10.1023/B:PHEN.0000041900.30172.e8
Rothkirch, M., Stein, T., Sekutowicz, M., and Sterzer, P. (2012). A direct oculomotor correlate of unconscious visual processing. Curr. Biol. 22, R514–R515. doi: 10.1016/j.cub.2012.04.046
Sand, A., and Nilsson, M. E. (2016). Subliminal or not? Comparing null-hypothesis and bayesian methods for testing subliminal priming. Conscious. Cogn. 44, 29–40. doi: 10.1016/j.concog.2016.06.012
Sandberg, K., Timmermans, B., Overgaard, M., and Cleeremans, A. (2010). Measuring consciousness: is one measure better than the other? Conscious. Cogn. 19, 1069–1078. doi: 10.1016/j.concog.2009.12.013
Schmidt, T. (2015). Invisible stimuli, implicit thresholds: why invisibility judgments cannot be interpreted in isolation. Adv. Cogn. Psychol. 11, 31–41.
Schmidt, T., and Vorberg, D. (2006). Criteria for unconscious cognition: three types of dissociation. Percept. Psychophys. 68, 489–504.
Schwiedrzik, C. M., Singer, W., and Melloni, L. (2011). Subjective and objective learning effects dissociate in space and in time. Proc. Natl. Acad. Sci. U.S.A. 108, 4506–4511. doi: 10.1073/pnas.1009147108
Sergent, C., and Dehaene, S. (2004). Is consciousness a gradual phenomenon? Evidence for an all-or-none bifurcation during the attentional blink. Psychol. Sci. 15, 720–728. doi: 10.1111/j.0956-7976.2004.00748.x
Shanks, D. R. (2016). Regressive research: the pitfalls of post hoc data selection in the study of unconscious mental processes. Psychon. Bull. Rev. doi: 10.3758/s13423-016-1170-y [Epub ahead of print].
Shanks, D. R., and John, M. F. S. (1994). Characteristics of dissociable human learning systems. Behav. Brain Sci. 17, 367–395. doi: 10.1017/S0140525X00035032
Shanks, D. R., Vadillo, M. A., Riedel, B., Clymo, A., Govind, S., Hickin, N., et al. (2015). Romance, risk, and replication: can consumer choices and risk-taking be primed by mating motives? J. Exp. Psychol. Gen. 144:e142-8. doi: 10.1037/xge0000116
Simmons, J. P., Nelson, L. D., and Simonsohn, U. (2011). False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol. Sci. 22, 1359–1366. doi: 10.1177/0956797611417632
Simmons, J. P., Nelson, L. D., and Simonsohn, U. (2012). A 21 Word Solution. Rochester, NY: Social Science Research Network.
Sklar, A. Y., Levy, N., Goldstein, A., Mandel, R., Maril, A., and Hassin, R. R. (2012). Reading and doing arithmetic nonconsciously. Proc. Natl. Acad. Sci. U.S.A. 109, 19614–19619. doi: 10.1073/pnas.1211645109
Stafford, T. (2014). The perspectival shift: how experiments on unconscious processing don’t justify the claims made for them. Front. Psychol. 5:1067. doi: 10.3389/fpsyg.2014.01067
Stein, T., Kaiser, D., and Hesselmann, G. (2016). Can working memory be non-conscious? Neurosci. Conscious. 2016:niv011. doi: 10.1093/nc/niv011
Street, C. N. H., and Vadillo, M. A. (2016). Can the unconscious boost lie-detection Accuracy? Curr. Dir. Psychol. Sci. 25, 246–250. doi: 10.1177/0963721416656348
Vadillo, M. A., Hardwicke, T. E., and Shanks, D. R. (2016a). Selection bias, vote counting, and money-priming effects: a comment on rohrer, pashler, and harris (2015) and vohs (2015). J. Exp. Psychol. Gen. 145, 655–663. doi: 10.1037/xge0000157
Vadillo, M. A., Konstantinidis, E., and Shanks, D. R. (2016b). Underpowered samples, false negatives, and unconscious learning. Psychon. Bull. Rev. 23, 87–102. doi: 10.3758/s13423-015-0892-6
Keywords: consciousness, awareness, unconscious processing, best practices, diversity
Citation: Rothkirch M and Hesselmann G (2017) What We Talk about When We Talk about Unconscious Processing – A Plea for Best Practices. Front. Psychol. 8:835. doi: 10.3389/fpsyg.2017.00835
Received: 01 February 2017; Accepted: 08 May 2017;
Published: 23 May 2017.
Edited by:
Nathan Faivre, École Polytechnique Fédérale de Lausanne, SwitzerlandReviewed by:
Zhicheng Lin, Ohio State University at Columbus, United StatesMiguel A. Vadillo, Universidad Autonoma de Madrid, Spain
Copyright © 2017 Rothkirch and Hesselmann. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Marcus Rothkirch, bWFyY3VzLnJvdGhraXJjaEBjaGFyaXRlLmRl Guido Hesselmann, Z3VpZG8uaGVzc2VsbWFubkBjaGFyaXRlLmRl