Regret Minimization and the Price of Total Anarchy

Regret Minimization and the Price of Total Anarchy Paper by A. Blum, M. Hajiaghayi, K. Ligett, A. Roth Presented by Michael Wunder

Nash Anarchy vs. Total Anarchy n n n In a multiagent setting, want to find the ratio between the socially optimal value and the “selfish” agent outcome Traditionally, assumed to be Nash, where no agent has incentive to change Can also find the price of total anarchy, when selfish agents act repeatedly to minimize regret over previous actions

Why Regret Minimization? n n n Finding Nash equilibria can be computationally difficult Not clear that agents would converge to it, or remain in one if there are several Regret minimization is realistic because there are efficient algorithms that minimize regret, it is locally computed, and players improve by lowering regret

Results comparing prices n n n Shows how Po. TA compares with Po. A Four classes of games Hotelling Games Valid Games Atomic Linear Congestion Games Parallel Link Congestion Games

Preliminaries (maximization) n n n Ai : set of pure strategies for player i Si : set of mixed strategies for player i (distributions over Ai ) Social Utility Function: Individual utility function: Strategy set if player i ’ changes from si to s’i:

Preliminaries (cont. ) n n n Socially Optimal Value: Regret of Player i given action sets A: The difference between action taken and best available action over all timesteps Price of Total Anarchy: Ratio of social value of best strategies to the “regret minimizers”

Hotelling Games n n Problem: k sellers must set up a vendor stand on a graph to sell to n tourists, who buy from first seller along a path Strategy set Ai = V S 1 S 2 T 1

Hotelling Games cont. n n Social welfare at time t: To maximize fairness (and maximize the lowest player), split all vertices equally OPT = n/k Si T 1

Hotelling Games cont. n Claim: Price of anarchy = (2 k-2)/k Proof: Consider alternate set: n Some player h achieves: n n n If player i plays same strategy as h, the expected payoff is: Therefore, Price of Anarchy

Hotelling with Total Anarchy n n n The price of total anarchy is also (2 k-2)/k Proof from symmetry: Let Oti be the set of plays at time t by players other than i Δit->u be the difference between expected payoff from choosing from Oti at time step u, and n/(2 k-2) For all i, for all 1<=t, u<=T: Δit->u + Δiu->t >=0 Imagine a (2 k-2) player game where there is a time t and a time u player for each original player but i If player i replaces a random player, αi = n/(2 k-2)

Hotelling Total Anarchy Proof n n If player i replaces a time t player, and all other time t players are removed, player i’s payoff only improves The expected payoff of player i from picking an action oti uniformly at random from Oti and playing over all T rounds:

Generalized Hotelling Games n n n The above proof does not use specifics of the game as described In general, Po. TA is (2 k-2)/k even in the presence of arbitrarily many Byzantine players making arbitrary decisions Regret-minimizing players may not converge to a Nash equilibrium, and play can cycle forever

Valid Games, Price of Anarchy n n n Valid games are a broad class of games that includes a market sharing game, the facility location problem, and others. Example: Cable television market sharing Game is bipartite graph G = ((V, U), E). Each v in V is a player, each u in U is a market Markets have value and cost Players have budget Players may enter adjacent markets, and receive value of market divided by players in market

Valid Games Definition n For a set function f, define the derivative of f at X in V in direction D in V-X to be f’D(X)=f(X U D)-f(X) A game is valid if: For X in A, γ i’(X)>= γ i’(A) for all i in V – A (submodularity) (Vickrey)

Valid Games Price of Anarchy n n n Vetta shows that for any Nash equilibrium strategy S, if γ is nondecreasing, γ(S) >= OPT/2 Po. TA matches Po. A While Po. A does not hold with the addition of Byzantine players, Po. TA does

Total Anarchy w/Byzantines So there is a regret minimizing player i which violates the regret minimizing condition.

Atomic Congestion Games n n n An atomic congestion game is a minimization game consisting of k players and a set of facilities V (ai over Vi) Each facility e has a latency function fe(le) Each player i has weight wi (unweighted wi = 1) Player i experiences cost: n load on facility le n

Atomic Congestion Games n n n Linear Edge Costs: Social utility: Consider two types of social utility function: linear and makespan in parallel link networks

Congestion Games Po. A n n Price of Anarchy with unweighted players, sum social utility function, and linear cost functions is 2. 5 (Christodoulou et al. 2005) Claim: Price of Total Anarchy is the same: “By assuming regret minimization, each player’s time average cost is no better than the cost of best action in hindsight. That is, no better than optimal strategy. ”

Congestion Games: Po. TA n n n Proof: for all i: Summing over all players: After math:

Congestion Games: Po. TA n For atomic congestion games with unweighted players, sum social function, and polynomial latency d 1 -o(1) functions of degree d, Po. TA <= d

Parallel Link Congestion Game n n identical links, k weighted players Each player pays sum of weights of jobs on link chosen Social cost is total weight of worst loaded link (makespan):

2 Parallel Links: Po. TA n For 2 links, Price of Total Anarchy matches Price of Anarchy = 3/2, but only in expectation

n Parallel Links: Po. TA n n With n parallel links, Po. TA is not the same as Po. A Po. TA with makespan utility and n links is Ω(n½), versus O(log n/ log n) for Po. A Proof: with n links and n players, OPT = 1 We can construct a situation with negative regret but with maximum latency = Ω(n½)

n Parallel Links: Po. TA n n n Divide the players into groups of size n½/2 and rotate each group to take link 1 The rest distribute evenly on the remaining links Each player has average latency 5/4 – ½ (n-½) If a player plays a fixed link, the average latency is 2 – ½ (n-½) Therefore, players have negative regret but maximum latency = Ω(n½)

Conclusion n Thank you!