Surprise, Curiosity, and Confusion Promote Knowledge Exploration: Evidence for Robust Effects of Epistemic Emotions

Research has started to acknowledge the importance of emotions for complex learning and cognitive performance. However, research on epistemic emotions has only recently become more prominent. Research in educational psychology in particular has mostly focused on examining achievement emotions instead of epistemic emotions. Furthermore, only few studies have addressed functional mechanisms underlying multiple different epistemic emotions simultaneously, and only one study has systematically compared the origins and effects of epistemic emotions with other emotions relevant to knowledge generation (i.e., achievement emotions; Vogl et al., 2019). The present article aimed to replicate the findings from Vogl et al. (2019) exploring within-person interrelations, origins, and outcomes of the epistemic emotions surprise, curiosity, and confusion, and the achievement emotions pride and shame, as well as to analyze their robustness and generalizability across two different study settings (online; Study 1, n = 169 vs. lab; Study 2, n = 79). In addition, the previous findings by Vogl et al. (2019, Study 3) and the present two new studies were meta-analytically integrated to consolidate evidence on origins and outcomes of epistemic emotions. The results of the two new studies largely replicated the findings by Vogl et al. (2019). Combined with the meta-analytic results, the findings confirm distinct patterns of antecedents for epistemic vs. achievement emotions: Pride and shame were more strongly associated with the correctness of a person’s answer (i.e., accuracy), whereas surprise, curiosity, and confusion were more strongly related to incorrect responses a person was confident in (i.e., high-confidence errors) producing cognitive incongruity. Furthermore, in contrast to achievement emotions, epistemic emotions had positive effects on the exploration of knowledge. Implications for research and practice are discussed.

Introduction

Although research has started to acknowledge the importance of emotions for complex learning and cognitive performance (see e.g., Angie et al., 2011), research on epistemic emotions, such as confusion, has only recently become of interest (e.g., D’Mello et al., 2014; Muis et al., 2015a; Arguel et al., 2019; Fayn et al., 2019; Vogl et al., 2019). In educational contexts in particular, scholars have focused predominantly on studying antecedents and performance effects of achievement emotions, such as anxiety, pride, or shame related to success and failure (Pekrun, 2018). However, epistemic emotions including surprise, curiosity, and confusion can also profoundly impact cognitive processing underlying learning and performance (Pekrun and Stephens, 2012; Muis et al., 2018a).

As such, gaining deeper understanding of the specific antecedents and effects of different emotions related to learning and knowledge generation is critically important for designing learning environments and intervention programs in a way that they foster adaptive emotions. However, the body of research for some epistemic emotions (e.g., confusion) is still sparse and ambiguous (e.g., D’Mello et al., 2014; Vogl et al., 2019). Vogl et al. (2019) is one of only a handful of extant studies addressing several epistemic emotions simultaneously to investigate their joint origins and effects (D’Mello and Graesser, 2012; Muis et al., 2015a, b, 2018b; Pekrun et al., 2017b; Trevors et al., 2017; Di Leo et al., 2019). Furthermore, to our knowledge, it is the first study that systematically compared origins and effects of these emotions with other emotions relevant to learning and knowledge generation (i.e., achievement emotions) and that used a within-person analytic approach to do so.

Given the novelty of the substantive research questions addressed by Vogl et al. (2019), as well as the lack of research addressing within-person emotional functioning, we aimed to test the robustness and the generalizability of the findings of their most extensive study (Study 3) across different settings (online Study 1 vs. lab Study 2) by conducting two direct replication studies. Direct replications are needed to test if a particular study design can reliably produce a specific empirical result (Schmidt, 2009). In addition, we used a meta-analytic approach to combine the findings of Study 3 from Vogl et al. (2019) and the results of both present studies to substantiate our knowledge on the distinct antecedents and effects of epistemic emotions. Meta-analysis increases statistical power as well as precision of estimates by pooling across studies, and thereby provides useful insights into the psychological phenomenon under study (e.g., Stanley et al., 2018). Replication and meta-analysis are both methods in line with the call for more generalizable research triggered by the “replication crisis” that has undermined the confidence in the robustness of psychological findings. Furthermore, both methods can be used to generate cumulative evidence required for developing sound, comprehensive theory that can inform practical perspectives on fostering adaptive emotions.

In contrast to the existing body of research that has predominantly focused on between-person designs (see e.g., Voelkle et al., 2014; Murayama et al., 2017), Vogl et al. (2019) used within-person analysis to examine functional relations between variables (Molenaar, 2004). Generally, empirical findings based on between-person data cannot be used to infer conclusions about the within-person relations proposed by emotion theories. While between-person approaches can be used to examine individual differences in the antecedents and effects of emotions, within-person analysis provides insight into the structures and variations of emotional states within individuals over time, in response to different task characteristics, for instance (e.g., Tsai et al., 2008; Murayama et al., 2011; Tanaka and Murayama, 2014). As such, within-person analysis is needed for gauging the validity of emotion theories that predict within-person relations between variables.

In sum, the present research seeks to provide robust evidence for a novel theoretical research area targeting antecedents and effects of an under-researched group of emotions. To do so, we used analytical methods that allowed us to derive reliable and valid conclusions about the origins and outcomes of epistemic emotions (within-person design, meta-analytic approach, and replication).

Origins of Epistemic and Achievement Emotions

Prototypically, epistemic emotions are triggered by cognitive incongruity, which can be produced by unexpected information that contradicts prior knowledge or personal beliefs, such as high-confidence errors (Marshall and Brown, 2006). Confrontation with unexpected information can interrupt ongoing cognitive processes and shift attention to this information. Enhanced processing and the exploration of this information can be a consequence. However, the specific effects may also depend on the emotions triggered by the cognitive incongruity (Vogl et al., 2019). Surprise may be the first reaction when one is confronted with unexpected or schema-incongruous information (Stiensmeier-Pelster et al., 1995; Meyer et al., 1997; Scherer, 2009; Noordewier and Breugelmans, 2013; Noordewier et al., 2016). In addition, curiosity (Loewenstein, 1994) and confusion (Pekrun and Stephens, 2012; Silvia, 2013) may be triggered, likely following surprise (Berlyne, 1954, 1960; Loewenstein, 1994; D’Mello and Graesser, 2012; Vogl et al., 2019).

However, situations involving cognitive incongruity may also elicit achievement emotions. Persons may also feel ashamed when something they were convinced of turns out to be incorrect, or proud if they are proven correct. More specifically, achievement- related shame is triggered by internally attributed failure (e.g., lack of ability; Weiner, 1985, 2010; Pekrun, 2018), whereas achievement-related pride is triggered by internally attributed success (e.g., ability or effort; Weiner, 1985, 2010; Tangney, 1999; Tracy and Robins, 2004, 2007; Pekrun, 2018). As such, achievement emotions differ from epistemic emotions in terms of their object focus, that is, achievement outcomes (Pekrun, 2018) vs. generation of knowledge (Brun et al., 2008).

Outcomes of Epistemic and Achievement Emotions

Epistemic emotions drive knowledge acquisition about the self and the world and are of critical importance for knowledge generation, conceptual change, and cognitive performance (Brun et al., 2008; Morton, 2010; Pekrun and Stephens, 2012; Muis et al., 2018a, b). More specifically, surprise can prompt interest, curiosity and confusion (Berlyne, 1954, 1960; Loewenstein, 1994; Renninger and Hidi, 2016; Vogl et al., 2019). It has been consistently found that curiosity promotes learning and achievement (Kang et al., 2009; von Stumm et al., 2011; Gruber et al., 2014; Marvin and Shohamy, 2016; Middlebrooks et al., 2016) as well as more specifically knowledge exploration (Berlyne, 1954, 1960; Litman et al., 2005). In contrast, effects of confusion on learning and knowledge exploration are inconsistent. Initial evidence suggests that confusion can promote learning and knowledge acquisition (Craig et al., 2004; D’Mello et al., 2014; Vogl et al., 2019) if cognitive incongruity can be resolved, as for example by engaging with the material (Brown and VanLehn, 1980; Mandler, 1990).

Similarly to confusion, findings on the effects of shame on learning and knowledge generation are inconsistent. For instance, shame has been associated with approach as well as avoidance tendencies (De Hooge et al., 2010), and it has been found to reduce intrinsic and amplify extrinsic motivation (Turner and Schallert, 2001). In contrast, pride in successful task performance has consistent positive effects on learning and achievement. Pride promotes task-oriented, extrinsic and intrinsic motivation, knowledge exploration, perseverences and effort, as well as academic achievement (Williams and DeSteno, 2008; Oades-Sese et al., 2014; Pekrun et al., 2017a, 2019).

Epistemic Emotions and Knowledge Exploration: Findings by Vogl et al. (2019)

In three independent experimental studies, Vogl et al. (2019) used a within-person analysis to investigate the origins and relations of the epistemic emotions surprise, curiosity, and confusion, and the achievement emotions pride and shame as well as their effects on knowledge exploration. This research question is novel and relevant for educational and emotion research in several ways: first, the authors focused on epistemic emotions, one group of emotions that is highly relevant for learning and knowledge generation but under-researched; secondly, they are one of the first researchers that analyzed common antecedents and effects of epistemic emotions and compared them to origins and outcomes of one other group of emotions relevant for learning and knowledge-generation (i.e., achievement emotions). And thirdly, they used a within-person approach to disentangle within- and between-person variance which allowed them to analyze the functional relations between the selected variables (Molenaar, 2004; Murayama et al., 2017). With that, their research results can be used to validate emotion theories (e.g., control-value theory of achievement emotions; Pekrun, 2018) and can inform practice about how to create emotionally sound learning environments.

Study 1 examined surprise, curiosity, and confusion and their effects on participants’ motivation to explore the correct answer in case an incorrect answer was given. Study 2 added two prototypicall achievement emotions, namely pride and shame, to the study design and investigated the effects of all five emotions on actual exploratory behavior in case of incorrect answers. Study 3 further expanded on Studies 1 and 2 by including a broader measure of exploratory behavior by allowing participants to request up to three pieces of information per statement. In addition, they investigated the effect of the five emotions on knowledge exploration after incorrect and correct answers. Since Study 3 offered the most comprehensive picture of the relations between accuracy and confidence, emotions, and exploratory behavior, the present paper aimed to replicate Study 3.

Using a trivia task with immediate feedback, Vogl et al. (2019) found that achievement emotions were more strongly associated with accuracy (i.e., correctness of the answer), whereas epistemic emotions were more strongly related to high-confidence errors (i.e., incorrect answers an individual is confident in) generating cognitive incongruity. Moreover, as compared with pride and shame, surprise, curiosity, and confusion were more strongly and positively related to the exploration of knowledge. Specifically, surprise and curiosity were positive predictors of knowledge exploration. Confusion had positive predictive effects on knowledge exploration which were significant in Studies 1 and 3 but not in Study 2. These results were largely consistent with theoretical considerations (Loewenstein, 1994; D’Mello and Graesser, 2012; Pekrun, 2018) with the exception of the variable results of confusion. Thus, they argued that the inconsistent findings for confusion are probably due to its weak effects on knowledge exploration and concluded that in particular the effects of confusion need to be investigated further.

Aims and Hypotheses

The first goal of this research was to directly replicate the findings of Study 3 from Vogl et al. (2019) to ensure that their study design can reliably produce the results found by Vogl et al. (2019) in different settings. Specifically, in Study 2, we collected data in a controlled lab setting instead of allowing participants to do the study online from home (Study 1). In this way, we controlled for confounding influences and distracting stimuli, such as searching the internet for the correct answer or interruptions during the study. In both studies, we used the exact same target group of participants, materials, and procedure as Vogl et al.’s (2019) in Study 3, with the exception of the different setting in Study 2 of this research. The second goal was to meta-analytically investigate the results of both present studies and the findings of Study 3 from Vogl et al. (2019) to gain a more precise understanding of the effects of the epistemic emotions and achievement emotions under study.

Both studies used a trivia task with immediate feedback to trigger surprise, curiosity, confusion, shame, and pride. We expected different patterns of antecedents for epistemic emotions and achievement emotions. In more detail, we hypothesized that epistemic emotions would be primarily produced by incorrect answers a person was confident in (i.e., high-confidence errors), whereas achievement emotions would be prompted by incorrect (shame) vs. correct (pride) answers (i.e., failure vs. success). As for the outcome of the emotions, we expected surprise, curiosity, and confusion to positively predict exploratory behavior. In addition, we hypothesized that surprise would not directly predict exploratory behavior but that this effects would be mediated by curiosity and confusion. In addition, we expected pride to promote exploratory behavior after correct answers. Due to the inconsistent findings for shame, we left the direction of the effect of shame on exploratory behavior open. We tested the following hypotheses (see Figure 1) supported by Vogl et al.’s (2019, Study 3) findings anew in the present replication studies:

Hypothesis 1: We expected surprise, curiosity, and confusion to be positively predicted by high-confidence errors.

Hypothesis 2: We expected that surprise would positively predict curiosity and confusion, and that curiosity and confusion would positively predict exploratory behavior. In addition, we hypothesized that curiosity and confusion would be mediators in this effect.

Hypothesis 3: We expected that high-confidence errors would positively predict exploratory behavior and that epistemic emotions would be mediators in the high-confidence error-exploration relation.

Hypothesis 4: We expected that accuracy would positively predict pride, and negatively predict shame.

Hypothesis 5: We expected that pride would positively predict exploratory behavior; we did not formulate a directional hypothesis for the effect of shame on exploratory behavior.

FIGURE 1

Figure 1. Main hypotheses.

Study 1

Study 1 aimed to directly replicate Study 3 from Vogl et al. (2019) using the exact same target group of participants, setting (online study), materials, procedure, and methods of analysis.

Method

Participants

The online study was completed by 169 (100 women, M_age = 25.53, SD = 7.46) from a German university. As thanks for their participation, they could win one of two €20 (US$23) gift cards.

Materials

Participants completed a task developed by Vogl et al. (2019) consisting of 20 items, including statements likely to trigger high-confidence errors because they target widespread errors in general knowledge to (e.g., “Ketchup is an American invention”) as well as statements with varying degrees of difficulty less likely to produce high-confidence errors (e.g., “Socrates was sentenced to die by drinking the hemlock cup”).

Procedure and Measures

For each of the 20 items, participants were asked to assess the accuracy of the statement and their confidence in the reply (6-point Likert scale; 1 = very uncertain to 6 = very certain). They got immediate feedback if their answer was correct (“Your answer is correct” vs. “Your answer is incorrect”) and were asked how surprised, curious, confused, ashamed, and proud they were currently feeling using a 5-point Likert scale (1 = not at all to 5 = very strong) for rating each emotion adjective (i.e., single-item versions of the Epistemic Emotion Scales; Pekrun et al., 2017b). Next, they had the opportunity to request an explanation of the statement (“Would you like to see the explanation now?” [No vs. Yes]) and up to two more pieces of information for each statement (“Would you like to receive more information concerning this topic?” [No vs. Yes]). Ethical approval was obtained by the Faculty of the first authors’ institution.

Data Analysis

Multilevel structural equation modeling was applied to distentangle within- and between-person relationships in the nested data (Level 1: trivia statements; Level 2: individuals) using Mplus 7.4 (type = twolevel; Muthén and Muthén, 2017). We tested two separate multilevel models:

First, we analyzed accuracy (0 = incorrect; 1 = correct), confidence, and the Accuracy × Confidence interaction as predictors of the emotions at Level 1 (Model 1) to test if high-confidence errors produce epistemic emotions (Hypothesis 1). We standardized accuracy and confidence before calculating the interaction term and centered the predictors within each individual to disentangle effects within- and between-persons (see Enders and Tofighi, 2007; Wang and Maxwell, 2015). At Level 2, variables simply correlated with each other. The model was saturated.

Secondly, we decomposed the Accuracy × Confidence interaction term to analyze its effects (Model 2). In Model 2a, we investigated the simple effects of confidence in incorrect answers (high-confidence errors) on the emotions and exploratory behavior. In Model 2b, we analyzed the same effects for correct answers. The epistemic emotions predicted exploratory behavior sequentially (Hypotheses 2 and 3): surprise predicted curiosity and confusion and both predicted exploratory behavior. 95% bootstrap confidence intervals were used to test the indirect effects. Again, variables simply correlated with each other at Level 2.

In line with recommendations by Hu and Bentler (1999), we interpreted the fit of Model 2 as good if values of the CFI were greater or equal to 0.95 and values of the RMSEA and SRMR were smaller or equal to 0.06, and as moderate if values of the CFI were between 0.90 and 0.95 and values of the RMSEA and SRMR were between 0.06 and 0.08. However, general cut-off values should be interpreted with caution (Marsh et al., 2004).

Results

Preliminary Findings

Descriptive statistics and correlations are displayed in Table 1. Scores sufficiently varied for all study variables. ICCs for emotions ranged from 0.185 to 0.339, indicating that within-person variation outweighed variation at the between-person level.

TABLE 1

Table 1. Estimated sample statistics (Means and SD) and within- and between-person correlations.

Origins of Epistemic and Achievement Emotions (Model 1)

Results for Model 1 are displayed in Tables 2, 3. Accuracy significantly negatively predicted surprise, curiosity, and confusion. In addition, the Accuracy × Confidence interaction was a significant negative predictor of all three epistemic emotions, confirming that high-confidence errors triggered epistemic emotions (Hypothesis 1).

TABLE 2

Table 2. Predictors of epistemic emotions (Model 1).

TABLE 3

Table 3. Predictors of achievement emotions (Model 1).

Furthermore, accuracy significantly positively predicted pride and significantly negatively predicted shame (Hypothesis 4). Additionally, the Accuracy × Confidence interaction term significantly positively predicted pride and significantly negatively predicted shame, indicating that participants were more proud in correct answers they were confident in and more ashamed in incorrect answers they were confident in. However, the interaction term predicted epistemic emotions more strongly than achievement emotions (β range −0.314 to −0.506 for the epistemic emotions and 0.116 and −0.126 for pride and shame; see Tables 2, 3, respectively).

Outcomes and Relations of Epistemic and Achievement Emotions (Model 2)

Model 2a (confidence in incorrect answers) fitted well to the data: χ²(1) = 1.223, p = 0.269; RMSEA = 0.012; CFI = 1.00; SRMR_within = 0.003. Model 2b (confidence in correct answers) showed an equally good fit: χ²(1) = 1.148, p = 0.284; RMSEA = 0.009; CFI = 1.00; SRMR_within = 0.003; see Figure 2 and Supplementary Table S1. In line with Hypothesis 1, surprise was significantly positively predicted by high-confidence errors and significantly negatively predicted by confidence in correct answers. In both models surprise significantly positively predicted curiosity and confusion. Confidence in correct answers significantly positively predicted pride but significantly negatively predicted shame. In contrast, high-confidence errors significantly positively predicted shame but were not significantly related to pride.

FIGURE 2

Figure 2. Results of Model 2 in Studies 1 (upper panel) and 2 (lower panel). inc, path coefficients for incorrect answers (Model 2a); cor, path coefficients for correct answers (Model 2b). Residuals and correlations between emotions are not depicted. ^∗p < 0.05, ^∗∗p < 0.01, ^∗∗∗p < 0.001.

In line with Hypothesis 2, curiosity significantly positively predicted exploratory behavior in both models. In contrast, confusion did not significantly predict exploratory behavior; however, in case of incorrect answers the path coefficient was positive and marginally significant (p = 0.060). Surprise had significant positive indirect effects on exploratory behavior that were mediated by curiosity but not by confusion in both models. However, the indirect effect mediated by confusion in case of incorrect answers was again marginally significant (p = 0.067). Overall, these findings provide partial support for the surprise-exploration relation hypothesis. In line with Hypothesis 5, pride positively predicted exploratory behavior after correct answers, whereas shame did not significantly relate to exploratory behavior.

In line with Hypothesis 3, high-confidence errors significantly positively predicted exploratory behavior. More specifically, we found significant indirect effects of confidence in incorrect answers on exploratory behavior that were mediated by surprise and curiosity but not by surprise and confusion. However, the confusion path was marginally significant (p = 0.066). Further supporting mediation, the direct effect of confidence in incorrect answers on exploratory behavior was not significant. In addition, there was a significantly indirect negative effect of confidence in correct answers on exploratory behavior mediated by surprise and curiosity but not by surprise and confusion. There was no significant direct effect of confidence in correct answers on exploratory behavior.

Discussion

Study 1 aimed to exactly replicate Vogl et al.’s (2019, Study 3) findings. We found the same patterns of antecedents for epistemic emotions and achievement emotions as Vogl et al. (2019): High-confidence errors positively predicted epistemic emotions, whereas success and failure predicted achievement emotions (Pekrun, 2018). However, the intensity of pride and shame depended also on participants’ confidence in their answer: High-confidence errors produced more shame than low-confidence errors. Conversely, participants experienced more pride in correct answers accompanied by high than low confidence. However, the Accuracy × Confidence interaction predicted epistemic emotions more strongly than achievement emotions.

In line with hypotheses, surprise and curiosity positively predicted knowledge exploration in case of incorrect and correct answers. The effects of surprise were mediated by curiosity. The significant positive effect of confusion on exploration in case of incorrect answers reported by Vogl et al. (2019, Study 3) was replicated in terms of the direction of the effect, but not in terms of significance. The path coefficient (ß = 0.062) and the 95% confidence interval (95% CI = [−0.003; 0.127]) indicate a positive trend. Furthermore, the confidence interval largely overlaps with the corresponding confidence interval produced by the previous findings (Study 3 in Vogl et al., 2019: 95% CI = [0.026, 0.174]).

In addition, replicating Vogl et al.’s (2019) findings, high-confidence errors positively predicted exploration. As expected, curiosity was a mediator in the effect of errors on exploration. Conversely, exploratory behavior seems to be underminded if confidence in the accuracy of one’s answer is confirmed as evidenced by the negative effects of confidence in correct answers on exploratory behavior. It is important to note, however, that the results are correlational. Experimental evidence is needed to confirm the proposed temporal order of the emotions.

As expected, pride after correct answers promoted exploratory behavior. Conversely, pride did not predict exploration after incorrect answers incorrect answers because participants did not experience pride in this case (M = 1.11, SD = 0.35). In line with Vogl et al.’s (2019) findings, shame did not significantly predict exploration. These findings support the assumption that shame can have variable effects and does not have to be detrimental for knowledge generating behaviors.

In sum, the findings from Vogl et al. (2019, Study 3) were directly replicated by Study 1 of the present research. In the next step, however, the generalizability of the findings needs to be tested. Therefore, Study 2 was moved to a different setting.

Study 2

Study 2 also aimed to directly replicate Study 3 from Vogl et al. (2019) by using the exact same target group of participants, materials, and procedure. However, we changed the study setting from online to lab to test the generalizability of the findings across these two settings. This study setting allowed also controlling for confounding effects and distracting stimuli.

Method

Participants

The lab study was completed by 78 participants (59 women, M_age = 23.62, SD = 7.13) from a German university. As thanks for their participation, they could choose between credit as a subject or €10.

Materials, Procedure, and Measures

Study 2 used the same materials, procedure, and measures as Study 1. However, the study was conducted in a laboratory under controlled conditions. Ethical approval has been obtained by the Faculty of the first authors’ institution.

Data Analysis

We used multilevel structural equation modeling with Mplus 7.4 (Muthén and Muthén, 2017) to model descriptive statistics for within- and between-person relations between the study variables. However, to estimate Models 1 and 2, we needed to reduce the number of parameters given smaller sample size to avoid model convergence errors in this study. As such, we used the type = complex-option to estimate these models. Therefore, Level 2 relations were not explicitly modeled.

Results

Preliminary Findings

Descriptive statistics and correlations are displayed in Table 1. Scores sufficient varied for all study variables. ICCs for emotions ranged from 0.157 to 0.323, indicating that within-person variation outweighed variation at the between-person level.

Origins of Epistemic and Achievement Emotions (Model 1)

Results for Model 1 are displayed in Tables 2, 3. The model was again saturated. Confirming high-confidence as antecedents epistemic emotions (Hypothesis 1) and replicating the Study 1 findings, accuracy and the Accuracy × Confidence interaction significantly negatively predicted surprise, curiosity, and confusion.

As expected, accuracy significantly positively predicted pride and significantly negatively predicted shame (Hypothesis 4). However, the Accuracy × Confidence interaction term also significantly positively predicted pride and significantly negatively predicted shame. These results indicate that participants were more proud in and less ashamed of correct answers they were confident in. However, the interaction term predicted epistemic emotions more strongly than achievement emotions (β range −0.324 to −0.527 for the epistemic emotions and 0.110 and −0.079 for pride and shame; see Tables 2, 3, respectively)

Outcomes and Relations of Epistemic and Achievement Emotions (Model 2)

Both models showed a good model fit with exception of the RMSEA: Model 2a (confidence in incorrect answers), χ²(1) = 8.360, p = 0.004; RMSEA = 0.095; CFI = 0.990; SRMR = 0.011; Model 2b (confidence in correct answers), χ²(1) = 34.206, p = 0.000; RMSEA = 0.212; CFI = 0.941; SRMR = 0.018 (see Figure 2 and Supplementary Table S1). As proposed in Hypothesis 1, high-confidence errors significantly positively predicted surprise, whereas confidence in correct answers significantly negatively predicted surprise. In both models surprise significantly positively predicted curiosity and confusion. In contrast to previous findings, pride was significantly positively predicted by confidence in correct as well as incorrect answers. As in Study 1, shame was significantly negatively predicted by confidence in correct answers.

As proposed in Hypothesis 2, curiosity significantly positively predicted exploration in both models, whereas confusion was not significantly related to exploratory behavior. The effect of surprise on exploratory behavior was significantly mediated by curiosity but not by confusion, after correct and incorrect answers. In contrast to Study 1 and Hypothesis 5, pride after correct answers did not significantly predict exploration. As expected, shame was not related to exploratory behavior.

In line with Hypothesis 3, high-confidence errors significantly positively predicted exploration. This indirect effect was significantly mediated by surprise and curiosity but not by surprise and confusion. The direct effects of confidence in incorrect and correct answers on exploration were not significant.

Discussion

Goal of Study 2 was to directly replicate Vogl et al.’s (2019, Study 3) findings and to test their generalizability across settings. However, we need to keep in mind that the settings addressed in the present studies are a lab situation and a particular online environment. Additional research is needed to test the generalizability of the findings across other settings that might be more representative of daily life (such as knowledge exploration in books or libraries or an unrestricted online search). In line with previous findings and the results of Study 1, epistemic emotions were produced by high-confidence errors, whereas the achievement emotions pride and shame were predominantly triggered by success and failure, respectively (Pekrun, 2018). Nevertheless, the intensity of pride and shame was also influenced by the confidence in one’s reply. However, as in Study 1 and Vogl et al. (2019), the Accuracy × Confidence interaction predicted epistemic emotions more strongly than achievement emotions.

Replicating previous findings, surprise and curiosity positively promoted exploration, following incorrect and correct answers. Curiosity was a mediator in this effect. As in Study 1, the positive effect of confusion on exploration was replicated in terms of the direction of the effect but not in terms of significance. The path coefficient (ß = 0.083) and the overlap of the 95% confidence interval with the confidence intervals of the previous findings indicated once more a trend supporting a positive direct effect of confusion on knowledge generation (Study 1: 95% CI = [−0.003; 0.127]; Study 2: 95% CI = [−0.029; 0.195]; Study 3 in Vogl et al. (2019): 95% CI = [0.026, 0.174]). The non-significant result in case of correct answers is not surprising because confusion is not experienced in this case (M = 1.19, SD = 0.58).

Furthermore, exploratory behavior was elicited by high-confidence errors. As expected, the epistemic emotions surprise and curiosity (but not confusion) were mediators in this effect. Exploratory behavior was not related to confidence in correct answers. Nevertheless, experimental research is needed to confirm the temporal order of epistemic emotions.

In contrast to Study 1 and Vogl et al.’s (2019, Study 3) results, the positive effect of pride after correct answers on knowledge exploration could not replicated neither in terms of the direction of the effect (ß = −0.114, p = 0.077) nor in terms of significance. The 95% confidence interval of the path (95% CI = [−0.240; 0.012]) is not overlapping with the previous findings indicating a positive trend (Study 1: 95% CI = [0.011; 0.112]; Study 3 in Vogl et al. (2019): 95% CI = [0.027, 0.144]). Clearly, more research is needed to understand the effect of pride after correct answers on knowledge exploration. As expected, incorrect answers did not result in pride (M = 1.06, SD = 0.20). Consequently, pride did not predict exploratory behavior in this case either. Finally, as in Study 1 and findings by Vogl et al. (2019), exploratory behavior was not related to shame, supporting the assumption that shame does not necessarily have to be detrimental for knowledge generating behaviors.

Meta-Analysis

Finally, we meta-analytically analyzed the estimates across the two present studies and Study 3 from Vogl et al. (2019) to get a more precise estimation of the effects and thereby a more accurate understanding of the origins, outcomes, and relations of the epistemic emotions surprise, curiosity, and confusion with the achievement emotions pride and shame.

Method

The package metafor (Viechtbauer, 2010) produced for the R environment (version 3.2.2; R Core Team, 2015) was used to conduct the meta-analysis including the results of Vogl et al. (2019, Study 3) and both studies of this research. For each path of Models 1 and 2, we ran separate random-effects models to calculate the mean weighted effects based on the findings from the three studies.

Results

Origins of Epistemic and Achievement Emotions (Model 1)

The mean weighted effect sizes ($\overline{\mathrm{ß}}$) for the predictors of epistemic emotions and achievement emotions are displayed in Tables 4, 5, respectively. The meta-analytic findings confirmed that the Accuracy × Confidence interaction significantly negatively predicted epistemic emotions ($\overline{\mathrm{ß}}$ range −0.337 to −0.515, Table 4). These findings support Hypothesis 1 claiming that epistemic emotions are triggered by high-confidence errors. In contrast, accuracy was the strongest predictor of achievement emotions (Hypothesis 4). Specifically, accuracy positively predicted pride ($\overline{\mathrm{ß}}$ = 0.594, Table 5) and negatively predicted shame ($\overline{\mathrm{ß}}$ = −0.375, Table 5). In addition, the meta-analytic findings showed that the Accuracy × Confidence interaction term also significantly predicted the experience of pride (positively) and shame (negatively). These findings indicate that the intensity of pride was also depended on one’s confidence in the answer. However, effect sizes for achievement emotions ($\overline{\mathrm{ß}}$ = 0.119 and −0.126 for pride and shame, respectively) were much weaker than for epistemic emotions and confidence intervals of the effects of epistemic and achievement emotions did not overlap (see Tables 4, 5).

TABLE 4

Table 4. Mean weighted effect sizes ($\overline{\mathrm{ß}}$) for epistemic emotions in Model 1 across three studies.

TABLE 5

Table 5. Mean weighted effect sizes ($\overline{\mathrm{ß}}$) for achievement emotions in Model 1 across three studies.

Outcomes and Relations of Epistemic and Achievement Emotions (Model 2)

Figure 3 and Supplementary Table S2 display the mean weighted effect sizes ($\overline{\mathrm{ß}}$ for direct effects, $\overline{b}$ for indirect effects) for Model 2. Meta-analytic findings support Hypothesis 1 by confirming that high-confidence errors were a strong significant positive predictor of surprise ($\overline{\mathrm{ß}}$ = 0.656). Conversely, surprise was significantly negatively predicted by confidence in correct answers. Curiosity and confusion were significantly positively predicted by surprise, after correct and incorrect answers. High-confidence errors significantly predicted pride but significantly negatively predicted shame. Vice versa, confidence in correct answers significantly positively predicted pride and significantly negatively predicted shame.

FIGURE 3

Figure 3. Mean weighted relations across three studies between confidence in answers, epistemic emotions, and exploration at the within-person level based on the meta-analytic findings. inc, path coefficients for incorrect answers (Model 2a); cor, path coefficients for correct answers (Model 2b). Residuals and correlations between emotions are not depicted. ^∗p < 0.05, ^∗∗p < 0.01, ^∗∗∗p < 0.001.

Furthermore, meta-analytic findings showed that curiosity and confusion significantly positively predicted exploratory behavior after incorrect answers (Hypothesis 2). Mean weighted effect sizes indicated a stronger effect of curiosity ($\overline{\mathrm{ß}}$ = 0.330) than confusion ($\overline{\mathrm{ß}}$ = 0.079). Curiosity also significantly predicted exploration after correct answers. In addition, exploratory behavior was significantly positively predicted by surprise. In more detail, the positive effects of surprise on exploration were mediated by curiosity and confusion, following incorrect answers. This mediation is additionally supported by the diminished direct effects of confidence in incorrect answers on curiosity ($\overline{\mathrm{ß}}$ = −0.026) and confusion ($\overline{\mathrm{ß}}$ = 0.141), respectively, as compared to Model 1. In case of correct answers, the significant positive indirect effect of surprise on exploration was mediated by curiosity only. Overall, these findings provide support for the surprise-exploration relation hypothesis (Hypothesis 2). In contrast to Hypothesis 5 and previous findings in the single studies, the meta-analytic findings did not support a positive effect of pride on exploratory behavior after correct answers. Exploratory behavior was not significantly predicted by shame either.

As expected, meta-analytic findings confirmed a significant positive effect of high-confidence errors on exploratory behavior mediated by surprise and curiosity as well as surprise and confusion (Hypothesis 3). In line with the direct effects, the mean weighted effect sizes indicated a stronger effect of surprise and curiosity ($\overline{\mathrm{ß}}$ = 0.089) than surprise and confusion ($\overline{\mathrm{ß}}$ = 0.020). The effect of confidence in correct answers on exploratory behavior was significantly mediated by surprise and curiosity only. In line with the mediation hypothesis, confidence in incorrect answers did not directly predict exploratory behavior. Confidence in correct answers significantly negatively predicted exploratory behavior.

Discussion

The meta-analysis allowed a more precise estimation of the effects, revealed consistency of our findings, and helped to resolve inconsistencies regarding the effect of confusion across the present studies and Study 3 from Vogl et al. (2019). Future research drawing on different stimuli, different methods to induce epistemic emotions, or different types of exploratory or knowledge generating behavior, for instance, is needed to further evaluate the generalizability of our findings. Overall, the meta-analytic results support distinct patterns of antecedents for epistemic emotions and achievement emotions: High-confidence errors elicit epistemic emotions (Hypothesis 1), whereas pride and shame are triggered more strongly by success and failure, respectively (Hypothesis 4; Pekrun, 2018). Interestingly, the results confirm that the intensity of achievement emotions also depends on confidence in the answer: Pride was more intensely experienced if confidence in the correct answers was high and shame was more intensely experienced if confidence in the incorrect answer was high. However, epistemic emotions were more strongly influenced by the interaction of accuracy and confidence than achievement emotions [range of the mean weighted effect sizes ($\overline{\mathrm{ß}}$ = −0.337 to −0.515 for the epistemic emotions and 0.119 and −0.126 for pride and shame); see Tables 4, 5, respectively].

In line with Hypothesis 2, surprise, curiosity, and confusion positively predicted knowledge exploration, after incorrect answers. Curiosity and confusion were mediators in the effect of surprise. As expected, exploratory behavior was positively predicted by high-confidence errors, while surprise and curiosity as well as surprise and confusion were mediators in this effect (Hypothesis 3). Conversely, exploratory behavior was reduced when participants’ expectations concerning the accuracy of their answer were confirmed.

With regard to the effect of pride after correct answers on exploration, the meta-analytic findings support Hypothesis 5 in terms of the direction of the effect, but not in terms of significance. However, the mean weighted effect size was very small ($\overline{\mathrm{ß}}$ = 0.024). Finally, shame did not significantly predict exploratory behavior. These findings suggest that shame may not necessarily be negative for knowledge generating behaviors.

General Discussion

Studies 1 and 2 largely replicated the findings by Vogl et al. (2019) and confirmed the robustness of their results and their generalizability across settings (online and lab), with the exception of the effects of confusion after incorrect answers and pride after correct answer on knowledge exploration. In addition, the meta-analysis allowed a more precise estimation of the effects. In sum, the results confirm distinct patterns of antecedents for epistemic and achievement emotions as well as positive effects of the epistemic emotions surprise, curiosity, and confusion on knowledge exploration.

Origins of Epistemic and Achievement Emotions

Both new studies conducted in this research largely replicated the findings from Vogl et al. (2019, Study 3). In addition, the meta-analysis supported that epistemic emotions surprise, curiosity, and confusion were elicited by cognitive incongruity induced by high-confidence errors (Berlyne, 1954; Loewenstein, 1994; D’Mello and Graesser, 2012; Pekrun and Stephens, 2012; Silvia, 2013), whereas achievement emotions were more strongly related to success and failure in the trivia task (i.e., answering a statement correctly vs. incorrectly; mean weighted effect sizes ($\overline{\mathrm{ß}}$) for pride and shame 0.594 and −0.375, respectively, Table 5; e.g., Pekrun et al., 2017a, b). Even though participants’ confidence in the answers also influenced the intensity of pride and shame, the impact was much stronger for epistemic emotions than for achievement emotions (range of the mean weighted effect sizes ($\overline{\mathrm{ß}}$) for the Accuracy × Confidence interaction predicting epistemic emotions −0.337 to −0.515, Table 4, and pride and shame 0.119 and −0.126, respectively, Table 5). In sum, these results confirm cognitive incongruity as “prime driver of epistemic emotions” (Vogl et al., 2019, p. 13) and highlight the connection of metacognitive processes and epistemic emotions.

The results further support the within-person relations of epistemic emotions as suggested by Vogl et al. (2019): the findings support surprise as an antecedent of curiosity and confusion (Loewenstein, 1994; D’Mello and Graesser, 2012). However, the relations observed in the three studies relevant for this research are of correlational nature. Experimental research is needed to test the causal relationship of surprise, curiosity, and confusion in more detail.

Outcomes of Epistemic and Achievement Emotions

The findings are in line with prior research on positive effects of curiosity and knowledge exploration (e.g., Litman et al., 2005) and support strong effects of curiosity on exploration after correct and incorrect answers. These effects were consistent across previous findings and both present studies. The results indicate that after incorrect answers cognitive incongruity triggers exploratory behavior mediated by surprise and curiosity. It is up for future research to investigate the origins of curiosity after correct answers. For example, personality traits may play an important role in the experience of state curiosity after correct and incorrect answers (e.g., epistemic beliefs or openness to experience; Muis et al., 2015a; Gocłowska et al., 2017; Fayn et al., 2019).

The findings for the effect of confusion on knowledge exploration after incorrect answers were less consistent across the three studies, including a significant positive effect in Study 3 from Vogl et al. (2019) and positive but non-significant effects in the present studies. However, confidence intervals of all three studies largely overlapped (Study 1: 95% CI = [−0.003; 0.127], Study 2: 95% CI = [−0.029;0.195], Study 3 from Vogl et al. (2019): 95% CI = [0.026, 0.174]). Finally, meta-analysis revealed a significant positive effect of confusion on knowledge exploration after incorrect answers ($\overline{\mathrm{ß}}$ = 0.079). This supports the interpretation of Vogl et al. (2019) that the differences in the effect sizes of the different single studies may have been caused by sampling error. Nevertheless, it should be kept in mind that the positive effects of confusion were small and that the possibility to explore information might be essential for positive effects of confusion on knowledge generation. It is up for future research to explore the effects of confusion in situations, in which exploration is not possible (e.g., exams). Therefore, future research needs to investigate the impact of confusion on other types of knowledge generating behaviors to test the robustness and generalizability of the findings.

Regarding achievement emotions, results were mixed. Study 1 replicated a significant positive effect of pride after correct answers on knowledge exploration (Study 1: β = 0.062, 95% CI = [0.011; 0.112]; Study 3 from Vogl et al. (2019): β = 0.085, 95% CI = [0.027; 0.144]), whereas Study 2 revealed a non-significant negative relationship between these variables (Study 2: β = −0.114, 95% CI = [−0.240; 0.012]). Taken together, meta-analysis indicated that pride after correct answers may not relate to knowledge exploration (meta-analysis: $\overline{\mathrm{ß}}$ = 0.024, 95% CI = [−0.084; 0.132]). Sampling error might explain the highly inconsistent findings across the three single studies. However, clearly more research is needed to investigate these contrary effects. The inclusion of more than three effect sizes in the meta-analysis will strengthen the overall power and with that the precision of the estimation of the effect. Nevertheless, it might be worthwhile investigating if the effects of pride differ in high-stakes achievement contexts, in which pride is usually studied (for an overview see Pekrun and Stephens, 2012), in contrast to low-stakes epistemic contexts as conceptualized in the present studies. In contrast to the findings for pride, we replicated the results for shame in both present studies. Meta-analytic findings additionally support the notion that shame did not impact knowledge exploration. This suggests in line theory and empirical evidence (Turner and Schallert, 2001; Pekrun and Stephens, 2012) that this negative emotion may not necessarily be detrimental for learning.

In sum, the findings confirm the robustness of the results from Vogl et al. (2019) and their generalizability across both settings (online and lab). They confirm high-confidence errors as predictors of epistemic emotions, and accuracy as predictor of achievement emotions. In addition, they support positive effects of all three epistemic emotions on the exploration of knowledge. In addition, the meta-analytic findings help to understand the effects of confusion on knowledge exploration and indicate a positive effect of this negative emotion on knowledge generation. However, the relationship between achievement emotions and epistemic behavior needs to be investigated in more detail.

Implications for Research and Practice

Generally, empirical findings based on between-person data cannot be used to infer conclusions about the within-person relations proposed by emotion theories. While a between-person approach can be used to examine individual differences in the antecedents and effects of emotions, within-person analyses provide an understanding of the structures and variations of emotional states within individuals over time in response to different task characteristics (e.g., Tsai et al., 2008; Murayama et al., 2011). As such, only within-person analyses can provide evidence on the validity of emotion theories that predict within-person relations between variables. Nevertheless, between-person designs still dominate the existing body of research (see, e.g., Voelkle et al., 2014; Murayama et al., 2017). We used within-person analyses to investigate relations, origins, and outcomes of epistemic and achievement emotions. Therefore, the present results directly support emotion theories on epistemic and achievement emotions (e.g., Scherer, 2009; D’Mello and Graesser, 2012; Pekrun, 2018) that describe psychological mechanisms within-persons and highlight the robustness of these effects.

The present research directly replicated the findings from Vogl et al. (2019, Study 3) and provides meta-analytic estimates of antecedents and effects of epistemic emotions based on a synthesis of three independent studies. Such integrative evidence is pivotal as it provides more precise estimates based on increased statistical power, and can be systematically expanded using evidence from additional replication studies. Conceptual replications are needed to further probe the generalizability of our findings and bolster our underlying theoretical assumptions. Conceptual replications test the robustness of the finding across a range of variable conditions by intentionally and systematically altering the design of the original study (e.g., Schmidt, 2009). With regard to the present research, the following steps may be particularly important: First, the robustness of the findings needs to be tested across different sets of stimuli to make sure that the effects are not triggered by inherent characteristics of the materials used in the studies (Westfall et al., 2015). In addition, methods to induce cognitive incongruity should be systematically altered in future studies. Based on previous research, confronting participants with contradictory information (Muis et al., 2015a) or with breakdown scenarios (D’Mello and Graesser, 2014) may provide viable alternative induction approaches for testing if cognitive incongruity will reliably trigger epistemic emotions. Furthermore, with regard to probing the robustness of effects of epistemic emotions, different indicators of knowledge exploration should be examined. Herein, indicators reflecting exploratory behaviors that are more common in daily life can also strengthen conclusions about the ecological validity of the effects. For example, knowledge exploration could be measured by tracing participants’ searches for information in an online learning environment, or by observing learners’ physical exploration of additional information in books or by seeking help from others. To be able to generalize the findings even more broadly, the role of epistemic emotions during a single lesson, a seminar, or even the whole course of studies needs to be analyzed to understand their total impact on knowledge generation. The present findings also have important applications for practice. The results indicate positive effects of positive and negative epistemic emotions: Curiosity and confusion can promote knowledge exploration. In addition, they shed light on the origins of emotions. This knowledge is important for designing practical interventions in educational contexts (e.g., school and university). For example, they suggest that epistemic emotions could be triggered in the classroom by challenging naïve theories to promote engagement with the learning material. Beyond the educational context, violation of expectations could be used more broadly to trigger epistemic emotions and to encourage persons to engage with information (i.e., surprising news headlines).

Data Availability Statement

The raw data supporting the conclusions of this manuscript will be made available by the authors, without undue reservation, to the reviewers upon request. The datasets for the present studies will be made publicly available in OSF upon acceptance of the manuscript (https://osf.io/dsx8f/).

Ethics Statement

The studies involving human participants were reviewed and approved by Research Ethics Committee of the Faculty of Psychology and Education of the University of Munich. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

EV, RP, KM, KL, and SS contributed to the conception and design of the study and manuscript revision, and read and approved the submitted version. EV and RP performed the statistical analysis. EV wrote the first draft of the manuscript. RP, KM, KL, and SS gave feedback and rewrote sections of the manuscript.

Funding

This work was supported by JSPS KAKENHI (Grant Numbers 16H06406 and 18H01102 to KM), F. J. McGuigan Early Career Investigator Prize from American Psychological Foundation to KM, and the Leverhulme Trust (Grant Numbers RPG-2016-146 and RL-2016-030 to KM).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2019.02474/full#supplementary-material

References

Angie, A. D., Connelly, S., Waples, E. P., and Kligyte, V. (2011). The influence of discrete emotions on judgement and decision-making: a meta-analytic review. Cogn. Emot. 25, 1393–1422. doi: 10.1080/02699931.2010.550751

PubMed Abstract | CrossRef Full Text | Google Scholar

Arguel, A., Lockyer, L., Chai, K., Pachman, M., and Lipp, O. V. (2019). Puzzle-solving activity as an indicator of epistemic confusion. Front. Psychol. 10:163. doi: 10.3389/fpsyg.2019.00163

PubMed Abstract | CrossRef Full Text | Google Scholar

Berlyne, D. E. (1954). A theory of human curiosity. Br. J. Psychol. Gen. Sect. 45, 180–191. doi: 10.1111/j.2044-8295.1954.tb01243.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Berlyne, D. E. (1960). Conflict, Arousal, and Curiosity. New York, NY: McGraw-Hill Book Company.

Google Scholar

Brown, J. S., and VanLehn, K. (1980). Repair theory: a generative theory of bugs in procedural skills. Cogn. Sci. 4, 379–426. doi: 10.1016/b978-1-4832-1446-7.50031-5

CrossRef Full Text | Google Scholar

Brun, G., Doğuoğlu, U., and Kuenzle, D. (eds). (2008). Epistemology and Emotions. Aldershot: Ashgate.

Google Scholar

Craig, S., Graesser, A., Sullins, J., and Gholson, B. (2004). Affect and learning: an exploratory look into the role of affect in learning with AutoTutor. J. Educ. Media 29, 241–250. doi: 10.1080/1358165042000283101

CrossRef Full Text | Google Scholar

De Hooge, I. E., Zeelenberg, M., and Breugelmans, S. M. (2010). Restore and protect motivations following shame. Cogn. Emot. 24, 111–127. doi: 10.1080/02699930802584466

PubMed Abstract | CrossRef Full Text | Google Scholar

Di Leo, I., Muis, K. R., Singh, C., and Psaradellis, C. (2019). Curiosity Confusion? Frustration! The role and sequencing of emotions during mathematics problem solving. Contemp. Educ. Psychol. 58, 121–137. doi: 10.1016/j.cedpsych.2019.03.001

CrossRef Full Text | Google Scholar

D’Mello, S., and Graesser, A. (2012). Dynamics of affective states during complex learning. Learn. Instr. 22, 145–157. doi: 10.1016/j.learninstruc.2011.10.001

CrossRef Full Text | Google Scholar

D’Mello, S., and Graesser, A. (2014). Confusion and its dynamics during device comprehension with breakdown scenarios. Acta Psychol. 151, 106–116. doi: 10.1016/j.actpsy.2014.06.005

PubMed Abstract | CrossRef Full Text | Google Scholar

D’Mello, S., Lehman, B., Pekrun, R., and Graesser, A. (2014). Confusion can be beneficial for learning. Learn. Instr. 29, 153–170. doi: 10.1016/j.learninstruc.2012.05.003

CrossRef Full Text | Google Scholar

Enders, C. K., and Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevel models: a new look at an old issue. Psychol. Methods 12, 121–138. doi: 10.1037/1082-989X.12.2.121

PubMed Abstract | CrossRef Full Text | Google Scholar

Fayn, K., Silvia, P. J., Dejonckheere, E., Verdonck, S., and Kuppens, P. (2019). Confused or Curious? openness/intellect predicts more positive interest-confusion relations. J. Pers. Soc. Psychol. 117, 1016–1033. doi: 10.1037/pspp0000257

PubMed Abstract | CrossRef Full Text | Google Scholar

Gocłowska, M. A., Baas, M., Elliot, A. J., and De Dreu, C. K. W. (2017). Why schema-violations are sometimes preferable to schema- consistencies: the role of interest and openness to experience. J. Res. Pers. 66, 54–69. doi: 10.1016/j.jrp.2016.12.005

CrossRef Full Text | Google Scholar

Gruber, M. J., Gelman, B. D., and Ranganath, C. (2014). States of curiosity modulate hippocampus-dependent learning via the dopaminergic circuit. Neuron 84, 486–496. doi: 10.1016/j.neuron.2014.08.060

PubMed Abstract | CrossRef Full Text | Google Scholar

Hu, L.-T., and Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct. Equ. Modeling 6, 1–55. doi: 10.1080/10705519909540118

CrossRef Full Text | Google Scholar

Kang, M. J., Hsu, M., Kajbich, I. M., Loewenstein, G., McClure, S. M., Wang, J. T., et al. (2009). The wick in the candle of learning: epistemic curiosity activates reward circuitry and enhances memory. Psychol. Sci. 20, 963–973. doi: 10.2139/ssrn.1308286

PubMed Abstract | CrossRef Full Text | Google Scholar

Litman, J., Hutchins, T., and Russon, R. (2005). Epistemic curiosity, feeling-of-knowing, and exploratory behaviour. Cogn. Emot. 19, 559–582. doi: 10.1080/02699930441000427

CrossRef Full Text | Google Scholar

Loewenstein, G. (1994). The psychology of curiosity: a review and reinterpretation. Psychol. Bull. 116, 75–98. doi: 10.1037/0033-2909.116.1.75

CrossRef Full Text | Google Scholar

Mandler, G. (1990). “Interruption (discrepancy) theory: Review and extensions,” in On the Move: The Psychology of Change and Transition, eds S. Fisher, and L. Cooper, (Chichester: Wiley), 13–32.

Google Scholar

Marsh, H. W., Hau, K.-T., and Wen, Z. (2004). In search of golden rules: comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Struct. Equ. Modeling 11, 320–341. doi: 10.1207/s15328007sem1103_2

CrossRef Full Text | Google Scholar

Marshall, M., and Brown, J. (2006). Emotional reactions to achievement outcomes: is it really best to expect the worst? Cogn. Emot. 20, 43–63. doi: 10.1080/02699930500215116

CrossRef Full Text | Google Scholar

Marvin, C. B., and Shohamy, D. (2016). Curiosity and reward: valence predicts choice and information prediction errors enhance learning. J. Exp. Psychol. Gen. 145, 266–272. doi: 10.1037/xge0000140

PubMed Abstract | CrossRef Full Text | Google Scholar

Meyer, W. U., Reisenzein, R., and Schützwohl, A. (1997). Toward a process analysis of emotions: the case of surprise. Motiv. Emot. 21, 251–274. doi: 10.1023/A:1024422330338

CrossRef Full Text | Google Scholar

Middlebrooks, C. D., McGillivray, S., Murayama, K., and Castel, A. D. (2016). Memory for allergies and health foods: how younger and older adults strategically remember critical health information. J. Gerontol. Ser. B Psychol. Sci. Soc. Sci. 71, 389–399. doi: 10.1093/geronb/gbv032

PubMed Abstract | CrossRef Full Text | Google Scholar

Molenaar, P. C. M. (2004). A manifesto on psychology as idiographic science: bringing the person back into scientific psychology, this time forever. Measurement 2, 201–218. doi: 10.1207/s15366359mea0204_1

CrossRef Full Text | Google Scholar

Morton, A. (2010). “Epistemic emotions,” in The Oxford Handbook of Philosophy of Emotion, ed. P. Goldie, (Oxford: Oxford University Press), 385–399.

Google Scholar

Muis, K. R., Chevrier, M., and Singh, C. A. (2018a). The role of epistemic emotions in personal epistemology and self-regulated learning. Educ. Psychol. 53, 165–184. doi: 10.1080/00461520.2017.1421465

CrossRef Full Text | Google Scholar

Muis, K. R., Sinatra, G. M., Pekrun, R., Winne, P. H., Trevors, G., Losenno, K., et al. (2018b). Main and moderator effects of refutation on value, epistemic emotions, and learning processes during knowledge revision. Contemp. Educ. Psychol. 55, 155–165. doi: 10.1016/j.cedpsych.2018.10.001

CrossRef Full Text | Google Scholar

Muis, K. R., Pekrun, R., Sinatra, G. M., Azevedo, R., Trevors, G., Meier, E., et al. (2015a). The curious case of climate change: testing a theoretical model of epistemic beliefs, epistemic emotions, and complex learning. Learn. Instr. 39, 168–183. doi: 10.1016/j.learninstruc.2015.06.003

CrossRef Full Text | Google Scholar

Muis, K. R., Psaradellis, C., Lajoie, S. P., Di Leo, I., and Chevrier, M. (2015b). The role of epistemic emotions in mathematics problem solving. Contemp. Educ. Psychol. 42, 172–185. doi: 10.1016/j.cedpsych.2015.06.003

CrossRef Full Text | Google Scholar

Murayama, K., Elliot, A. J., and Yamagata, S. (2011). Separation of performance-approach and performance-avoidance achievement goals: a broader analysis. J. Educ. Psychol. 103, 238–256. doi: 10.1037/a0021948

CrossRef Full Text | Google Scholar

Murayama, K., Goetz, T., Malmberg, L.-E., Pekrun, R., Tanaka, A., and Martin, A. J. (2017). “Within-person analysis in educational psychology: Importance and illustrations,” in British Journal of Educational Psychology Monograph Series II: Psychological Aspects of Education – Current Trends: The Role of Competence Beliefs in Teaching and Learning, eds D. W. Putwain, and K. Smart, (Oxford: Wiley), 71–87.

Google Scholar

Muthén, L. K., and Muthén, B. O. (2017). Mplus. User’s Guide, 8th Edn. Los Angeles, CA: Muthén & Muthén.

Google Scholar

Noordewier, M. K., and Breugelmans, S. M. (2013). On the valence of surprise. Cogn. Emot. 27, 37–41. doi: 10.1080/02699931.2013.777660

PubMed Abstract | CrossRef Full Text | Google Scholar

Noordewier, M. K., Topolinski, S., and Van Dijk, E. (2016). The temporal dynamics of surprise. Soc. Pers. Psychol. Compass 10, 136–149. doi: 10.1111/spc3.12242

CrossRef Full Text | Google Scholar

Oades-Sese, G. V., Matthews, T. A., and Lewis, M. (2014). “Shame and pride and their effects on student achievement,” in International Handbook of Emotions in Education, eds R. Pekrun, and L. Linnenbrink-Garcia, (New York, NY: Routledge), 246–264.

Google Scholar

Pekrun, R. (2018). “Control-value theory: A social-cognitive approach to achievement emotions,” in Big Theories Revisited 2: a Volume of Research on Sociocultural Influences on Motivation and Learning, eds G. A. D. Liem, and D. M. McInerney, (Charlotte, NC: Information Age Publishing), 162–190.

Google Scholar

Pekrun, R., Lichtenfeld, S., Marsh, H. W., Murayama, K., and Goetz, T. (2017a). Achievement emotions and academic performance: longitudinal models of reciprocal effects. Child Dev. 88, 1653–1670. doi: 10.1111/cdev.12704

PubMed Abstract | CrossRef Full Text | Google Scholar

Pekrun, R., Vogl, E., Muis, K. R., and Sinatra, G. M. (2017b). Measuring emotions during epistemic activities: the epistemically-related emotion scales. Cogn. Emot. 31, 1268–1276. doi: 10.1080/02699931.2016.1204989

PubMed Abstract | CrossRef Full Text | Google Scholar

Pekrun, R., Murayama, K., Marsh, H. W., Goetz, T., and Frenzel, A. C. (2019). Happy fish in little ponds: testing a reference group model of achievement and emotion. J. Pers. Soc. Psychol. 117, 166–185. doi: 10.1037/pspp0000230

PubMed Abstract | CrossRef Full Text | Google Scholar

Pekrun, R., and Stephens, E. J. (2012). “Academic emotions,” in APA Educational Psychology Handbook: Individual Differences and Cultural and Contextual Factors. APA Handbooks in Psychology, Vol. 2, eds K. R. Harris, S. Graham, T. Urdan, S. Graham, J. M. Royer, and M. Zeidner, (Washington, DC: American Psychological Association), 3–31.

Google Scholar

R Core Team (2015). R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing.

Google Scholar

Renninger, K. A., and Hidi, S. (eds). (2016). The Power of Interest for Motivation and Learning. New York, NY: Routledge.

Google Scholar

Scherer, K. R. (2009). The dynamic architecture of emotion: evidence for the component process model. Cogn. Emot. 23, 1307–1351. doi: 10.1080/02699930902928969

CrossRef Full Text | Google Scholar

Schmidt, S. (2009). Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Rev. Gen. Psychol. 13, 90–100. doi: 10.1037/a0015108

CrossRef Full Text | Google Scholar

Silvia, P. J. (2013). Knowledge Emotions: Feelings That Foster Learning, Exploring, and Reflecting. Available at: http://nobaproject.com/chapters/knowledge-emotions-feelings-that-fosterlearning-exploring-and-reflecting (accessed January 29, 2014).

Google Scholar

Stanley, T. D., Carter, E. C., and Doucouliagos, H. (2018). What meta-analyses reveal about the replicability of psychological research. Psychol. Bull. 144, 1325–1346. doi: 10.1037/bul0000169

PubMed Abstract | CrossRef Full Text | Google Scholar

Stiensmeier-Pelster, J., Martini, A., and Reisenzein, R. (1995). The role of surprise in the attribution process. Cogn. Emot. 9, 5–31. doi: 10.1080/02699939508408963

CrossRef Full Text | Google Scholar

Tanaka, A., and Murayama, K. (2014). Within-person analyses of situational interest and boredom: interactions between task-specific perceptions and achievement goals. J. Educ. Psychol. 106, 1122–1134. doi: 10.1037/a0036659

CrossRef Full Text | Google Scholar

Tangney, J. P. (1999). “The self-conscious emotions: Shame, guilt, embarrassment and pride,” in Handbook of Cognition and Emotion, eds T. Dalgleish, and M. J. Power, (New York, NY: Wiley), 541–568. doi: 10.1002/0470013494.ch26

CrossRef Full Text | Google Scholar

Tracy, J. L., and Robins, R. W. (2004). Putting the self into self-conscious emotions: a theoretical model. Psychol. Inq. 15, 103–125. doi: 10.1207/s15327965pli1502_01

CrossRef Full Text | Google Scholar

Tracy, J. L., and Robins, R. W. (2007). The psychological structure of pride: a tale of two facets. J. Pers. Soc. Psychol. 92, 506–525. doi: 10.1037/0022-3514.92.3.506

PubMed Abstract | CrossRef Full Text | Google Scholar

Trevors, G. J., Muis, K. R., Pekrun, R., Sinatra, G. M., and Muijselaar, M. M. L. (2017). Exploring the relations between epistemic beliefs, emotions, and learning from texts. Contemp. Educ. Psychol. 48, 116–132. doi: 10.1016/j.cedpsych.2016.10.001

CrossRef Full Text | Google Scholar

Tsai, Y.-M., Kunter, M., Lüdtke, O., Trautwein, U., and Ryan, R. M. (2008). What makes lessons interesting? The role of situational and individual factors in three school subjects. J. Educ. Psychol. 100, 460–472. doi: 10.1037/0022-0663.100.2.460

CrossRef Full Text | Google Scholar

Turner, J. E., and Schallert, D. L. (2001). Expectancy-value relationships of shame reactions and shame resiliency. J. Educ Psychol. 93, 320–329. doi: 10.1037//0022-0663.93.2.320

CrossRef Full Text | Google Scholar

Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. J. Stat. Softw. 36, 1–48. doi: 10.1186/s13643-019-1118-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Voelkle, M. C., Brose, A., Schmiedek, F., and Lindenberger, U. (2014). Towards a unified framework for the study of between-person and within-person structures: building a bridge between two research paradigms. Multivariate Behav. Res. 49, 193–213. doi: 10.1080/00273171.2014.889593

PubMed Abstract | CrossRef Full Text | Google Scholar

Vogl, E., Pekrun, R., Murayama, K., and Loderer, K. (2019). Surprised–curious–confused: epistemic emotions and knowledge exploration. Emotion doi: 10.1037/emo0000578 [Epub ahead of print].

CrossRef Full Text | PubMed Abstract | Google Scholar

von Stumm, S., Hell, B., and Chamorro-Premuzic, T. (2011). The hungry mind: intellectual curiosity is the third pillar of academic performance. Perspect. Psychol. Sci. 6, 574–588. doi: 10.1177/1745691611421204

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, L., and Maxwell, S. E. (2015). On disaggregating between-person and within-person effects with longitudinal data using multilevel models. Psychol. Methods 20, 63–83. doi: 10.1037/met0000030

PubMed Abstract | CrossRef Full Text | Google Scholar

Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychol. Rev. 92, 548–573. doi: 10.1037//0033-295X.92.4.548

CrossRef Full Text | Google Scholar

Weiner, B. (2010). The development of an attribution-based theory of motivation: a history of ideas. Educ. Psychol. 45, 28–36. doi: 10.1080/00461520903433596

CrossRef Full Text | Google Scholar

Westfall, J., Judd, C. M., and Kenny, D. A. (2015). Replicating studies in which samples of participants respond to samples of stimuli. Perspect. Psychol. Sci. 10, 390–399. doi: 10.1177/1745691614564879

PubMed Abstract | CrossRef Full Text | Google Scholar

Williams, L. A., and DeSteno, D. (2008). Pride and perseverance: the motivational role of pride. J. Pers. Soc. Psychol. 94, 1007–1017. doi: 10.1037/0022-3514.94.6.1007

PubMed Abstract | CrossRef Full Text | Google Scholar