Donghua University Emergence of cooperation through coevolving time

Donghua University Emergence of cooperation through coevolving time scale in spatial prisoner’s dilemma Zhihai Rong (荣智海) rongzhh@gmail. com Donghua University 2010. 08@The 4 th China-Europe Summer School on Complexity Science, Shanghai © 2002 IBM Corporation

DHU Donghua University Acknowledgements ØDr. Zhi-Xi Wu ØDr. Wen-Xu Wang ØDr. Petter Holme ü Zhi-Xi Wu, Zhihai Rong & Petter Holme, Phys. Rev. E, 036106, 2010 ü Zhihai Rong, Zhi-Xi Wu & Wen-Xu Wang, Phys. Rev. E, 026101, 2010 2

DHU Donghua University 阿豺折箭 戮力一心 Ø阿豺有子二十人。阿豺谓曰：“汝 等各奉吾一支箭。”折之地下。俄 而命母弟慕利延曰：“汝取一支箭 折之。”慕利延折之。又曰：“汝取 十九支箭折之。”延不能折。阿豺 曰：“汝曹知否？单者易折，众则 难摧，戮力一心，然后社稷可固 ！” ——《魏书 • 吐谷浑传》 3

DHU Donghua University Cooperation: the basis of human societies Robert Boyd and Sarah Mathew, A Narrow Road to Cooperation, SCIENCE, 2007 4

DHU Donghua University Prisoner’s dilemma (囚徒困境, PD) ØCooperator: help others at a cost to themselves. ØDefector: receive the benefits without providing help. C C D D (-2, -2) (-5, -1) (-1, -5) (-3, -3) Whatever opponent does, player does better by defecting… 5

DHU Donghua University Some rules for evolutions cooperation Nowak MA (2006). Five rules for the evolution of cooperation. Science ØKin selection: relative Hamilton, J. Theor. Biol. 7 (1964) ØDirect reciprocity: unrelated individuals Tit for tat(TFT): nice, punishing, forgiving, but for noise… Axelrod & Hamilton, Science 211, (1981) Win stay, lost shift(WSLS) Nowak, Sigmund, Nature 364, (1993) ØIndirect reciprocity: reputation Nowak, Sigmund, Nature 437 (2005). ØNetwork reciprocity 6

DHU Donghua University Spatial Game Theory M. Nowak and R. May, Evolutionary games and spatial chaos, Nature 1992 ØEach player x poccupying a site on a network pplaying game with neighbors and obtaining payoff: Px(t) pupdating rule( replicator dynamics): select a neighbor and learn its behavior with probability ~ f(Py(t)-Px(t)) 7

DHU Donghua University Evolutionary games on graphs G. Szabo&G. Fath, Evolutionary games on graphs, Phys. Rep. 446, 2007 ØCooperator frequency fc Game Rule Evolutionary Rule Replacement rule Selection rule ØBest take over Øreplicator dynamics W(x y) =f(Py-Px) ØRandom ØFermi dynamics: ØPreferential W(x y)=(1+exp(x-y/κ))-1 … ØWin stay, lost shift ØMemory … 8 PD, SG, SH, UG, PGG, Rock-paper-scissors… Structure & property ØLattice, random graph, small-world, scale-free… Ø, γ, rk , CC, community

DHU Diversity of lifetime (time scale) Donghua University C. Roca, J. Cuesta, A. Sánchez (2006), Physical review letters, vol. 97, pp. 158701. Z. X. Wu, Z. H. Rong, P. Holme (2009), Physical Review E, vol. 80, pp. 36106. ØThe interaction time scale — how frequently the individuals interact with each other ØThe selection time scale — how frequently they modifies their strategies ØThe selection time scale is slower than the interaction time scale, the player has a finite lifetime. ØIndividuals local on a square lattice. ØThe fitness of i at t-th generation: fi(t)=afi(t-1)+(1 -a)gi , where -- gi is the payoff of i -- a characterizes the maternal effects. ØWith probability pi, an individual i is selected to update its strategy: where κ characterizes the rationality of individuals, and is set as 0. 01. Ø 1/pi is the lifetime of i’s current strategy, f(0)=1. 9

DHU Donghua University Some key quantities to characterize the cooperative behaviors ØFrequency of cooperators: fc ØThe extinction threshold of defectors/cooperators: bc 1 and bc 2 10 All. D C & D coexist All. C

DHU Monomorphic time scale University Donghua a↗ fc ↗ Optimal fc occurs at p=0. 1 for a=0. 9 Øp 1, C is frequently exploited by D. ØP 0, Ds around the boundary have enough time to obtain a fitness high enough to beat Cs. ØCoherence resonance ü M. Perc, New J. Phys. 2006, M. Perc & M. Marhl, New J. Phys. 2006 ü J. Ren, W. -X. Wang, & F. Qi, Phys. Rev. E 75, 2007 11

DHU Polymorphic time scale Donghua University ØThe leaders are the individual with low p Øthe followers are the individual with high p. Øv% of individuals’ p are 0. 1, and others’ p are 0. 9. v=0. 5, a=0. 9, b=1. 1, fc ≈0. 7 12

DHU Donghua University Coevolving time scale Z. H. Rong, Z. X. Wu, W. X. Wang, Emergence of cooperation through coevolving time scale in spatial prisoner's dilemma, submitted to Physical Review E , 82, 026101 , 2010 Ø“win-slower, lose-faster” rule: i updates its strategy by comparing with neighbor j with a different strategy with probability üIf i successfully resists the invasion of j, the winner i is rewarded by owing longer lifetime: pi=pi-β, where β is reward factor üIf i accepts j's strategy, the loser i has to shorten its lifetime: pi=pi+α, where α is punishment factor Ø 0. 1 ≤ pi≤ 1. 0, initially pi=1. 0, κ=0. 01 ØWhat kind of social norm parameters (α, β) can promote the mergence of cooperation? 13

ØHigh time scale C(p>0. 5) High time scale D(p>0. 5) DHU Donghua ØLow time scale C (p≤ 0. 5) Low time scale D(p ≤ 0. 5) University The extinction threshold of cooperators, r. D a (α, β)=(0. 9, 0. 9) Long-term D cluster (α, β)=(0. 9, 0. 1) Long-term C cluster (α, β)=(0. 9, 0. 05) short-term C cluster 14 (α, β)=(0. 0, 0. 1) (α, β)=(0. 2, 0. 1)

DHU α=0, increasing β(reward) University Donghua ØInitially p=1, pmin=0. 1 Ø High time scale C High time scale D Ø Low time scale C Low time scale D t=100 15 t=50000

ØHigh time scale C High time scale D DHU ØLow time scale C Low time scale D Donghua University a (α, β)=(0. 9, 0. 1) 16 (α, β)=(0. 0, 0. 1) (α, β)=(0. 2, 0. 1)

DHU β =0. 1, increasing α(punishment) Donghua University Øα↗, fc↗ ØFeedback mechanism for C/D: ü Winner C fc↗ fintess↗ ü Winner D fc↘ fintess↘ Øα↗, their losing D neighbors have greater chance to becoming C, hence cooperation is promoted. 17 (α, β)=(0. 9, 0. 1) b=1. 05

ØHigh time scale C High time scale D DHU ØLow time scale C Low time scale D a Donghua University (α, β)=(0. 9, 0. 9) (α, β)=(0. 9, 0. 1) (α, β)=(0. 9, 0. 05) 18 (α, β)=(0. 0, 0. 1) (α, β)=(0. 2, 0. 1)

DHU α =0. 9, increasing β(reward) (α, β)=(0. 9, 0. 9) Donghua University (α, β)=(0. 9, 0. 1) (α, β)=(0. 9, 0. 05) 19

DHU Donghua University Coevolution of Teaching activity A. Szolnoki and M. Perc, New J. Phys. 10 (2008) 043036 A. Szolnoki, et al. , Phys. Rev. E 80(2009) 021901 Ø The player x will adopt the randomly selected neighbor y’s strategy with: Ø wx characterizes the strength of influence (teaching activity) of x. The leader with wx 1. ØEach successful strategy adoption process is accompanied by an increase in the donor’s teaching activity: If y succeeds in enforcing its strategy on x, wy wy+Δw. ØA highly inhomogeneous distribution of influence may emerge. 20

DHU Donghua University Multiplicative “win-slower, lose-faster” Ø“win-slower, lose-faster” rule: i updates its strategy by comparing with neighbor j with a different strategy: üIf i successfully resists the invasion of j, the winner i is rewarded by owing longer lifetime: pi=max(pi/β, pmin) üIf i accepts j's strategy, the loser i has to shorten its lifetime: pi=min(pi*α, pmax) üpmin=0. 1 and pmax=1. 0 21 The extinction threshold of cooperators, r. D

DHU Donghua University The extinction threshold of cooperation ØFor loser: α↗ ØFor winner: β mid ØThe additive-increase /multiplicative-decrease (AIMD) algorithm in the TCP congestion control on the Internet Jacobson, Proc. ACM SIGCOMM' 88 22 The extinction threshold of cooperators, r. D

DHU Donghua University Conclusions ØThe selection time scale is slower than the interaction time scale. ØBoth the fixed and the coevolving time scale. Ø“win-slower, lose-faster” rule ØThe potential application in the design of consensus protocol in multi-agent systems. 23

DHU Donghua University 东华大学 http: //cist. dhu. edu. cn/index. asp Ø东华大学位于上海松江区，原名中国纺织大学，是国家 教育部所属的211全国重点大学，也是我国首批具有博士、 硕士、学士三级学位授予权的大学之一。 Ø信息学院现有“控制理论与控制 程(90)”和“模式识别与 智能系统(02)” 2个博士点以及7个硕士点，“控制科学与 程(03)”一级学科博士后流动站，拥有“教育部数字化 纺织服装技术 程研究中心”。 Ø信息学院现有教职 近 120人，其中校特聘教授2人，长 江特聘讲座教授1人，博士生导师16人，具有正高级职称 25人，副高级职称 41人。 24

DHU Donghua University THANKS! Discussing Rong Zhihai (荣智海)：rongzhh@gmail. com Department of Automation, DHU 25