Exploring norms indispensable for both emergence and maintenance of cooperation in indirect reciprocity

Indirect reciprocity is one of the major mechanisms of the evolution of human cooperation. In indirect reciprocity, social norms with which individuals distinguish good people from bad people play essential roles. On the one hand, previous studies have suggested the various different norms which are evolutionarily stable against the invasion of free riders. However, these approaches could not reveal what norms would be selected in the process of evolution of cooperation because they are based on the premise that a single norm is shared in a society. On the other hand, recent studies have tackled the mechanisms of the coevolution of norms and cooperation. However, the norms which are necessary for emergence or sustenance of cooperation have not been revealed. Here, we show some indispensable norms for emergence and sustenance of cooperation using a norm knockout method which has been developed to analyze a function of each norm in the environment of coexistence of many norms. The results revealed that norms known as “shunning” and “image scoring” are indispensable in the emergence of cooperation but they are not required after a cooperative society is achieved. Furthermore, “simple standing” is a unique norm which is necessary to maintain cooperation. We call the former as a pioneer norm and the latter as a keystone norm. The results indicate importance to focus on the dynamics of evolution because the role of indispensable norms has been overlooked by the static analysis of evolutionarily stable norms.

1 Introduction

As a proverb says that one good turn deserves another, the evolution of a costly cooperative behavior can be understood as a mechanism in which a positive evaluation of the present cooperation is shared with others and the cooperating person receives cooperation from a third person at a future time. This mechanism is generally referred to as indirect reciprocity and numerous studies have addressed this mechanism [1, 5, 19, 23, 30]. In order for the aforementioned mechanism to work as the basis of cooperation, it is necessary for people to have norms that effectively distinguish good people from bad people and to have a tendency cooperating only toward the good people. In previous studies of indirect reciprocity, researchers have focused on several pro-social norms, such as “cooperation is good and defection is bad [6, 12]” (image scoring, “IS”), “cooperating toward bad person is bad and defecting toward bad person is good [3, 16]” (stern judging, “SJ”), and “cooperation even toward bad person is good and defection toward bad person is also good [2, 6, 18]” (simple standing, “SS”).

Most theoretical studies on indirect reciprocity have explored norms that are evolutionally stable. Theoretically, assuming that the same norm is adopted by all members of a population, Ohtsuki and Iwasa [11, 14] have shown that only eight out of 4,096 norms can stabilize cooperation. Here, “stable” means that the corresponding population which adopts the same norm cannot be invaded by other action rules. Although their approach allows for identifying norms that can sustain cooperation, it cannot answer what norms will evolve in a mixed situation of various norms. Additionally, in the empirical approaches, it has been explored what kind of norms people have in an indirect reciprocal situation [4, 9, 32]. The earlier approaches have pointed out that image scoring [6, 12] to judge whether people cooperate by past actions of others has been adopted. In recent years, more detailed experiments have been conducted and it has been found that the information referred to by individuals is varied [15, 16]. However, few studies have elucidated how norms are accepted by groups.

To reveal the evolution of cooperation in an environment in which various norms coexist as in a real society, competition among multiple norms must be taken into consideration. By coexisting of norms originally shared in each group or society, the conflict of norms has been causing fatal social problems, for example, pro-social behavior seen from one side is judged as anti-social behavior from the other and then they intensely conflict with each other. Especially in recent society, interaction between communities with divergent values is becoming more active and members are coming and going more intensely. Popularization of the Internet and becoming borderless among countries has accelerated this trend. The issue on how to construct a cooperative society in an environment in which diverse norms coexist and sometimes conflict is an urgent problem in modern society. Therefore, it is necessary to model the environment in which multiple norms coexist and compete in society as a norm ecosystem and clarify the coevolutionary process of norms and cooperation.

Studies focusing on competition among multiple norms have been limited to those dealing with competition among a few norms [17, 26]. On the other hand, recent other studies have also revealed the mechanism which enables to evolve cooperation in an environment where many norms coexist [19]. Systems where many norms coexist are complex in their interactions, making it difficult to understand the characteristics of individual norms. A previous study [20] proposed a new method for analyzing the role of a particular norm. The method removes a particular norm from a group in which multiple norms coexist and observe the group’s behavior to estimate the norm’s role. The method is called a norm knockout method, a variation on a technique known as gene knockout, which is used in genetic engineering.

In the norm ecosystem, two perspectives are needed to analyze the process of cooperation evolving and stabilizing. One is whether indispensable norms exist to evolve cooperative behavior from a coexistence state of norms or a state where defection is dominant. A previous study [20] has focused on this point. In the ecosystem, there is a species, called as a pioneer species, which first takes root in a devastated environment and contributes to the development of the subsequent ecosystem [21]. For example, plants that initially invade lands covered with lava by volcanoes are typical pioneer species. If this species does not exist, it is difficult for ecosystems such as forests to grow. Metaphorically applying pioneer species in the norm ecosystem, the previous study [20] can be regarded as a discovery of pioneer norms.

The other is whether indispensable norms exist to maintain that state after the cooperative society has been achieved. Little is known about norms indispensable for stabilizing cooperation in competition among various norms. In the ecosystem, there is a biological species called as a keystone species. This species greatly affects the ecosystem despite not being in a significant position in the food chain [10, 22]. A keystone species refers to a species that, when removed from a stable ecosystem, collapses ecosystems. For instance, sea otters in the kelp forest of the sea and beavers building dams in rivers are typical keystone species. In this study, similar to the relationship between the pioneer species and the pioneer norms, we focus on the indispensable norms to maintain cooperation after the cooperative society has been achieved. In other words, this study reveals a keystone norm in the norm ecosystem. In addition, by comparing the pioneer and keystone norms, we clarify what role various norms play in the evolution and stability of cooperation.

2 Models

In this study, we model finite individuals who play as donors and recipients in the giving game with no spatial structures. The group size of the model is N. The overview of the giving game is as follows. 1) One donor (i) and one recipient (j) are chosen at random from the population and they engage in a giving game. 2) The donor decides whether to cooperate with the recipient at a personal cost c (>0). If the donor chooses to cooperate, the recipient receives a benefit b > c; otherwise, the recipient obtains nothing. The donor chooses cooperation or defection with a recipient using an image that the donor has to the recipient. An image is either Good or Bad. If a donor’s image to a recipient is Good, the donor cooperates with the recipient and if the image of the recipient is Bad, the donor chooses defection. 3) Subsequently, N − 1 agents other than the donor, as observers, evaluate and update images of the donor using their own norms. Each agent independently has images of other agents, and thus images from each agent to others can be represented as an image vector. These image vectors form an N × N matrix, A. Every element of A is either Good or Bad. All agents consider themselves as Good, and thus all diagonal elements of A are set to Good constantly. Observer k updates donor i’s image (A_ki) depending on an action of donor i (C/D), an image of recipient j from viewpoint of observer k (A_kj) and a norm observer k adopts. In the case of the recipient being an observer (j = k), the recipient updates the donor’s image using his/her norm as well as those of other observers’ cases. Note that any updated donor’s images in the eyes of different observers are not necessarily same. Figure 1 shows the overview illustration of the game.

FIGURE 1

FIGURE 1. Overview of the game.

In this study, the norm of an agent is denoted as one of four possible “assessment combinations” and there are two possible “alleles (Good/Bad)” at the “locus” for each of the four assessment combinations. This combination of Good/Bad is the norm held by the agent. There is a total of 2⁴ = 16 possible norms. The first locus of the gene represents an assessment rule for an agent who cooperates with a good recipient. The second locus represents an assessment rule for an agent who cooperates with a bad recipient. The third locus represents an assessment rule for an agent who defects with a good recipient. The fourth locus represents an assessment rule for an agent who defects with a bad recipient. The new image to i from k’s viewpoint depends on i’s action (C/D) and depends on the image to the recipient j from the viewpoint of k (Good/Bad). Table 1 shows an example of a typical norm which have pro-social characteristics. SH evaluates a donor as Good only when the donor cooperates with a good recipient. In other words, a donor who is matched with a bad recipient will consistently be rated Bad. SJ evaluates cooperation with a bad recipient (unjustified cooperation) as Bad and defection against a bad recipient (justified defection) as Good. IS evaluates cooperation as Good and defection as Bad, regardless of whether the recipient is Good or Bad. SS evaluates only defection against good recipients as Bad and all other cases as Good. These norms have two common characteristics, cooperation with a good recipient is Good, and defection against a good recipient is Bad.

TABLE 1

TABLE 1. Typical norms which have pro-social characteristics.

Each simulation runs throughout G generations. Each agent plays the giving game R times as donor in each generation. At the beginning of each generation, an image of each agent from all of other agents (including themselves) is set to Good. Each player evolves one’s norm S_i using a pseudo-genetic algorithm [24] at the end of each generation. The pseudo-genetic algorithm adopted in this study follows an evolutionary-computation method that represents norms as sequence of numbers (genes) and applies genetic operations such as selection, mutation, and crossover to obtain a norm that has a high degree of fitness. In the model, fitness is represented as the accumulated payoff in a generation. In the process of evolution, each agent randomly selects two agents from N agents (including itself) to become its parents. For choosing parents, we adopt a roulette selection method. This roulette selection sets a probability distribution of all agents as ${\mathrm{\Pi }}_i={(U_i-U_{\mathit{min}})}^2/{\sum }_j{(U_j-U_{\mathit{min}})}^2$, where U_i denotes the agent i’s accumulated payoff in a generation given by U_i = bW − cV, with W being the number of donations i received in the generation and V the number of donations i gave. U_min means a minimum value of the accumulated payoffs among all agents. Finally, each agent updates its norm using a uniform crossover technique. With mutation rate (m), each locus is inverted to maintain the diversity of the norm space.

Each agent has two different types of errors: one, the probability that the agent’s updating of its evaluation of others, Good/Bad, is inverted (errors in perception), described as p, and two, the probability to perform an action different from the one prescribed by its action rule (errors in implementation), described as q. The implementation error is bilateral. State variables and initialization in the simulation are shown in Table 2. Generation G is set to 1,000 in the analyses of pioneer norms. Also, G is set to 2,000 in the analyses of keystone norms and the norm knockout method for a focal norm is executed at the 1,000th generation. The reason for executing the norm knockout at the 1,000th generation is that a cooperative regime is achieved and stabilized in all replications (50 replications) at the 1000th generation. We can observe the dynamics after the focal norm is removed from a cooperative society.

TABLE 2

TABLE 2. State variables and initialization in the simulation.

The norm knockout method is implemented as follows. When we knockout a particular norm, that norm is removed in the first round of each generation. Concretely, if the norm of an agent evolves into a norm that is knocked out as a result of the adopting process, the norm of the agent is changed to one of the other 15 norms randomly. In other words, the norm that is knocked out will never exist at all in a society.

The analysis of pioneer norms is executed as follows. The norm knockout method is used in the first generation. As a result, there are 15 norms except for the knocked-out norm in the population. This operation enables us to observe which norms are indispensable for the evolution of cooperation from the melting pot of norms. On the other hand, the analysis of keystone norms is executed as follows. In the first generation of the simulation, all the possible norms exist in the population. Cooperation can evolve in the situation which exists all 16 norms [20]. After cooperation is achieved and kept stable, at the 1,000th generation, we adopt the norm knockout method. This operation allows us to observe whether cooperation could be sustained when the focal norm is removed from the population.

3 Results

The purpose of this study is to clarify the keystone norms. For this purpose, we first discuss the character of keystone norms by conducting a pioneer norm analysis found in previous studies [20] and comparing it to a keystone norm analysis. Figure 2 shows the results of pioneer norms analysis and keystone norms analysis. In the results of the pioneer norms analysis, GBBB(SH), GGBB(IS), and GGBG (SS) are pioneer norms as same as in the previous study. Cooperation cannot evolve in an environment in which these norms do not exist. On the other hand, the results of the keystone norms analysis show that cooperation collapsed only when GGBG (SS) was knocked out. In other words, a keystone norm is only SS. SH and IS are pioneer norms but not keystone norms.

FIGURE 2

FIGURE 2. Results of pioneer norms analysis and keystone norms analysis. It shows the average cooperation ratio in the final generation when a representative norm was knocked out. The number of trials is 50 and error bars represent standard deviation. In pioneer norms analysis, when SH, IS, and SS are knocked out, cooperation does not evolve, whereas, in keystone norms analysis, only SS is necessary to maintain cooperation and cooperation is maintained even if SH and IS are knocked out. In addition, there is no effect even if IS is removed from a cooperative society, but cooperation also cannot evolve in the environment in which IS does not exist from the beginning.

As shown in Figure 3A, we reveal that cooperation cannot evolve when SH is knocked out in the pioneer norms analysis. In an environment in which SH does not exist, cooperation never evolves because defection is dominant at the beginning and no other norm can invade BBBB(ALLB). On the other hand, as shown in Figure 3B, cooperation is stable in the results of the keystone norms analysis. Although SH is knocked out at 1,000 generations, cooperation maintains robustly in a coexisted situation of four norms of GGGG (ALLG), GGGB, GGBG (SS), and GGBB(IS) which are cooperative norms.

FIGURE 3

FIGURE 3. Pioneer norms analysis (A) and keystone norm analysis (B) regarding SH. The x-axis shows generations and the y-axis shows the fraction of norms and cooperation ratio. The solid line represents population of each norm and the black dotted line represents cooperation ratio. In pioneer norms analysis, SH is knocked out at 0 generations and in keystone norms analysis, SH is knocked out at 1,000 generations. In pioneer norms analysis, cooperation does not evolve because SH is knocked out and also no other norm can invade ALLB. After cooperation evolves, SH cannot exist. Therefore, even if SH is knocked out in keystone norms analysis, cooperation is maintained stably.

On the other hand, as shown in Figures 4A,B, when SS is knocked out, cooperative society cannot realize in both pioneer norms analysis and keystone norms analysis. In pioneer norms analysis, when SS is knocked out from the beginning of the simulation, although SH is dominant over ALLB, cooperation cannot evolve from that state to a stable situation because SH assesses only cooperation toward good agents as Good, it does not cooperate by assessing all other things as Bad when other norms cooperate or defect toward bad agents. Furthermore, SH does not cooperate with other SH agents who have failed to cooperate because of errors. As a result, mutual defection among SH increases and cooperation never evolves among SH. Although SJ and IS exist, these norms are also vulnerable to errors and cannot maintain a stable cooperative state.

FIGURE 4

FIGURE 4. Pioneer norms analysis (A) and keystone norms analysis (B) regarding SS. The x-axis shows generations and the y-axis shows the fraction of norms and cooperation ratio. In pioneer norms analysis, SS is knocked out at 0 generations and in keystone norms analysis, SS is knocked out at 1,000 generations. When SS is knocked out in both pioneer norms analysis and keystone norms analysis, the cooperative state is not realized.

Additionally, in keystone norms analysis, even when SS is knocked out in a state in which cooperation is stable, cooperation collapses. ALLG is vulnerable to the invasion of defective norms such as ALLB because it cooperates unconditionally. GGGB is a norm that assesses defection toward good agents as Good, and it cannot contribute to maintaining cooperation. Since IS is vulnerable to errors as aforementioned, there is no other norm that can maintain cooperation in the absence of SS.

Subsequently, we analyze the cases where all 16 kinds of norms including BBBB and GGGG were knocked out. Table 3 shows the results of pioneer norms analysis and keystone norms analysis in the cases where all 16 kinds of norms were knocked out. In pioneer norms analysis, we can clarify that SH, IS, and SS are pioneer norms. In keystone norms analysis, we can clarify that only SS is a keystone norm. Although GGBB(IS), GGGB, and GGGG (ALLG) coexist with GGBG (SS) in a cooperative society, a cooperative society never collapses even if other than SS are knocked out. In other words, when we observe the cooperative society obtained as a result of evolution, the coexistence of these four kinds of norms is observed but only one norm is necessary for maintaining cooperation. Also, when we observe cooperative society, SH does not exist. However, cooperative society is not realized in an environment in which SH does not exist from the beginning.

TABLE 3

TABLE 3. Average cooperation ratios at the final generation. Figures between parentheses indicate standard deviations. It is highlighted in red that the cooperation ratio falls below 0.2. It shows that average of cooperation ratio after 1,000 generations in pioneer norms analysis and average of cooperation ratio after 2000 generations in keystone norms analysis. In both the analyses, the number of replications is 50.

4 Conclusion

We analyzed the norms necessary for cooperation based on indirect reciprocity to evolve. In the previous research studies, because they had a premise that a single norm is shared in a society, it was not possible to analyze the mechanism which enables to evolve cooperation in environments with various norms coexisted. Therefore, we developed the model where various norms coexist in a society as a norm ecosystem and analyzed the evolutionary mechanism of cooperation by agent-based simulation.

Applying the norm knockout method [20], we revealed two types of norms which are indispensable to evolve or maintain cooperation. One is the norm that is indispensable for cooperation to evolve from an environment in which defection is dominant and the other is the norm that is indispensable to maintain cooperation in an environment in which cooperation is dominant. SH, IS, and SS correspond to the former and SS corresponds to the latter. It is worth noting that SS is an indispensable norm in both cases. Some previous studies have revealed that SS can generate high levels of cooperation [8, 13, 25]. These results indicate that SS might be the keystone norm. On the other hand, a recent study has suggested that SS has problems when defectors are predominant [27]. However, no mechanism has been established to explain why SS can play an indispensable role in both defective and cooperative regimes. Understanding specific behavior and the relationship among multiple norms remain for future studies.

Although the two norms, especially SH and IS, are indispensable to emerge and evolve cooperative behaviors in a society in which defection is dominant, they are unable to survive or only a small number can exist in a society in which cooperation is stable [20], and these norms are called as unsung hero norms. Surprisingly, the keystone norms analysis revealed that unsung hero norms do not affect cooperation in a society in which cooperation is stable even if it is completely removed.

This is thought to give important suggestions for maintaining a cooperative social system. In other words, SH and IS are norms that do not contribute to a society when they are observed in a state in which cooperation is stable. Therefore, these norms may be excluded as not being adaptive. However, when a society becomes a defective state, it is impossible to rebuild a cooperative society without these norms. By analyzing the norm ecosystem, it was suggested that maintaining the diversity of norms in a society has an important meaning for a stable and sustainable cooperative society. Moreover, SJ was found to be neither a pioneer norm nor a keystone norm. In the study of [20] an environment with no errors, SJ temporarily accounted for a majority in the evolutionary process of cooperation and played an important role in a transition from a defective society to a cooperative society. However, it has been pointed out that SJ is vulnerable to errors [28] and cannot play an important role in the evolution and maintenance of cooperation in environments with both errors and coexistence of diverse norms because SJ assesses cooperation toward bad (pampering) as Bad and defection toward bad (punishment) as Good, it is a norm that uses secondary information most precisely. The role of SJ in the norm ecosystem is an issue to be explored in the future.

In this study, we examined the norm that combines two types of information: donors’ behaviors and recipients’ reputations. On the other hand, in the experimental studies, it has been discussed whether people actually use two types of information [4, 9, 25]. Although the information processing cost is ignored in this model when evaluating others, we need to analyze a more realistic norm ecosystem by introducing costs to the use of reputation as future development. Furthermore, models that introduce structures into the scope of interaction and learning have also been proposed; however, research studies have been limited to the studies dealing only with image scoring [7, 29] and to the models in which only three of pure defection, pure cooperation, and target norms exist in spite of dealing with multiple norms [31]. It is also a future issue to expand a norm ecosystem on a complex network. This study deals only with donor behavior (first-order information) and recipient image (second order information). However, some literature deals with donor image (third-order information) [12] or higher-order information [32]. A norm ecosystem that adopts higher-order information should be considered in the future.

Data availability statement

The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

Author contributions

HY initiated and performed the project. HY, IO, SU, and TS designed the project and approved the submission. HY, IO, SU, and TS wrote the manuscript. All authors reviewed the manuscript.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors, and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Alexander R. The biology of moral systems. New York: Aldine de Gruyter (1987).

Google Scholar

2. Sugden R. The economics of rights, cooperation and welfare. Oxford: Basil Blackwell (1986).

Google Scholar

3. Kandori M. Social norms and community enforcement. Rev Econ Stud (1992) 59:63–80. doi:10.2307/2297925

CrossRef Full Text | Google Scholar

4. Wedekind C, Milinski M. Cooperation through image scoring in humans. Science (2000) 288:850–2. doi:10.1126/science.288.5467.850

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Panchanathan K, Boyd R. Indirect reciprocity can stabilize cooperation without the secondorder free rider problem. Nature (2004) 432:499–502. doi:10.1038/nature02978

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Nowak MA, Sigmund K. Evolution of indirect reciprocity by image scoring. Nature (1998) 393:573–7. doi:10.1038/31225

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Nowak MA, Sigmund K. The dynamics of indirect reciprocity. J Theor Biol (1998) 194:561–74. doi:10.1006/jtbi.1998.0775

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Pacheco JM, Santos FC, Chalub FAC. Stern-judging: A simple, successful norm which promotes cooperation under indirect reciprocity. Plos Comput Biol (2006) 2:e178. doi:10.1371/journal.pcbi.0020178

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Leimar O, Hammerstein P. Evolution of cooperation through indirect reciprocity. Proc R Soc Lond B (2001) 268:745–53. doi:10.1098/rspb.2000.1573

CrossRef Full Text | Google Scholar

10. Panchanathan K, Boyd R. A tale of two defectors: The importance of standing for evolution of indirect reciprocity. J Theor Biol (2003) 224:115–26. doi:10.1016/S0022-5193(03)00154-1

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Ohtsuki H, Iwasa Y. How should we define goodness? -reputation dynamics in indirect reciprocity. J Theor Biol (2004) 231:107–20. doi:10.1016/j.jtbi.2004.06.005

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Ohtsuki H, Iwasa Y. The leading eight: Social norms that can maintain cooperation by indirect reciprocity. J Theor Biol (2006) 239:435–44. doi:10.1016/j.jtbi.2005.08.008

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Milinski M, Semmann D, Bakker TCM, Krambeck H-J. Cooperation through indirect reciprocity: Image scoring or standing strategy? Proc R Soc Lond B (2001) 268:2495–501. doi:10.1098/rspb.2001.1809

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Yamamoto H, Suzuki T, Umetani R. Justified defection is neither justified nor unjustified in indirect reciprocity. PLOS ONE (2020) 15:e0235137. doi:10.1371/journal.pone.0235137

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Swakman V, Molleman L, Ule A, Egas M. Reputation-based cooperation: Empirical evidence for behavioral strategies. Evol Hum Behav (2016) 37:230–5. doi:10.1016/j.evolhumbehav.2015.12.001

CrossRef Full Text | Google Scholar

16. Okada I, Yamamoto H, Sato Y, Uchida S, Sasaki T. Experimental evidence of selective inattention in reputation-based cooperation. Sci Rep (2018) 8:14813. doi:10.1038/s41598-018-33147-x

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Uchida S, Sigmund K. The competition of assessment rules for indirect reciprocity. J Theor Biol (2010) 263:13–9. doi:10.1016/j.jtbi.2009.11.013

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Uchida S. Effect of private information on indirect reciprocity. Phys Rev E (2010) 82:036111. doi:10.1103/PhysRevE.82.036111

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Uchida S, Yamamoto H, Okada I, Sasaki T. A theoretical approach to norm ecosystems : Two adaptive architectures of indirect reciprocity show different paths to the evolution of cooperation. Front Phys (2018) 6:14. doi:10.3389/fphy.2018.00014

CrossRef Full Text | Google Scholar

20. Yamamoto H, Okada I, Uchida S, Sasaki T. A norm knockout method on indirect reciprocity to reveal indispensable norms. Sci Rep (2017) 7:44146. doi:10.1038/srep44146

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Swaine MD, Whitmore TC. On the definition of ecological species groups in tropical rain forests. Vegetatio (1988) 75:81–6. doi:10.1007/BF00044629

CrossRef Full Text | Google Scholar

22. Paine RT. A note on trophic complexity and community stability. Am Nat (1969) 103:91–3. doi:10.1086/282586

CrossRef Full Text | Google Scholar

23. Mills LS, Doak DF. The keystone-species concept in ecology and conservation. Bioscience (1993) 43:219–24. doi:10.2307/1312122

CrossRef Full Text | Google Scholar

24. Holland J. Adaptation in natural and artificial systems. Michigan: Univ. of Michigan Press (1975).

Google Scholar

25. Brandt H, Sigmund K. Indirect reciprocity, image scoring, and moral hazard. Proc Natl Acad Sci U S A (2005) 102:2666–70. doi:10.1073/pnas.0407370102

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Martinez-Vaquero LA, Cuesta JA. Evolutionary stability and resistance to cheating in an indirect reciprocity model based on reputation. Phys Rev E (2013) 87:052810. doi:10.1103/PhysRevE.87.052810

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Hilbe C, Schmid L, Tkadlec J, Chatterjee K, Nowak MA. Indirect reciprocity with private, noisy, and incomplete information. Proc Natl Acad Sci U S A (2018) 115:12241–6. doi:10.1073/pnas.1810565115

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Uchida S, Sasaki T. Effect of assessment error and private information on stern-judging in indirect reciprocity. Chaos Solitons Fractals (2013) 56:175–80. doi:10.1016/j.chaos.2013.08.006

CrossRef Full Text | Google Scholar

29. Peleteiro A, Burguillo JC, Chong SY. Exploring indirect reciprocity in complex networks using coalitions and rewiring. In: Proc. 2014 int. Conf. Auton. Agents multi-agent syst. (2014). p. 669–76.

Google Scholar

30. Liu A, Wang L, Zhang Y, Sun C. Coevolution of cooperation and complex networks via indirect reciprocity. In: Liu D, Xie S, Li Y, Zhao D, and El-Alfy E-SM, editors. Neural information processing. Cham: Springer International Publishing (2017). p. 919–26.

CrossRef Full Text | Google Scholar

31. Sasaki T, Yamamoto H, Okada I, Uchida S. The evolution of reputation-based cooperation in regular networks. Games (2017) 8:8. doi:10.3390/g8010008

CrossRef Full Text | Google Scholar

32. Santos FP, Santos FC, Pacheco JM. Social norm complexity and past reputations in the evolution of cooperation. Nature (2018) 555:242–5. doi:10.1038/nature25763

PubMed Abstract | CrossRef Full Text | Google Scholar