Influence and Noise Gil Kalai Hebrew University of

Influence and Noise Gil Kalai Hebrew University of Jerusalem, Yale University Microsoft R&D, Israel ICS 2011, Beijing January 2011 1

Part I: Influence 2

Cause When does event A cause event B? Example (Kira Radinsky’s automatic system for making deductions based on internet searches): Earthquake causes Tsunami Cabbage grew causes Linux release 3

Causality When does event A cause event B? Central problem in philosophy, law, economics, physics, statistics, CS… (Examples are often somber) Two people try to assassin a third person who plan a trip to the desert. One puts poison in his jar, the other empties it. A person throws a baby from a tall building, another is waiting with a sharp sword. 4

Influence The word “influence” (dating back, according to Merriam-Webster dictionary, to the 14 th century) is close to the word “fluid”. The original definition of influence is: “an ethereal fluid held to flow from the stars and to affect the actions of humans. ” The modern meaning (according to Wictionary) is: ”The power to affect, control or manipulate something or someone. ” 5

Influence What is the influence an event A has on another event B? Can be regarded as an approach to causality and also as a generalization. 6

Influence (Of a variable on a function of many variables) The “amount” that changing the value of a variable will change the value of the function. 7

Boolean Functions We consider a BOOLEAN FUNCTION f : {-1, 1}n {-1, 1} f(x 1 , x 2, . . . , xn) It is convenient to regard {-1, 1}n as a probability space with the uniform probability distribution. 8

Influence We consider a BOOLEAN FUNCTION f : {-1, 1}n {-1, 1} The influence of the kth variable xk on f, denoted by Ik(f) is the probability that flipping the value of the kth variable will flip the value of f. I(f) is the sum of all individual influences. 9

Examples 1) Dictatorship f(x 1 , x 2, . . . , xn) =x 1 Ik(f) = 0 for k>1 I 1(f)=1 2) Majority f(x 1 , x 2, . . . , xn) =1 iff x 1 + x 2+. . . +xn > 0 Ik(f) behaves like n-1/2 for every k. 10

Critical Percolation 11

Examples (cont. ) 3) The crossing event for percolations For percolation, every hexagon corresponds to a variable. xi =-1 if the hexagon is white and xi =1 if it is grey. f=1 if there is a left to right grey crossing. Ik(f) behaves like n-3/8 for every k but few. 12

Examples (cont. ) 4) Recursive majority of threes Ik(f) behaves like n-log 3 for every k. 5) Ben-Or Linial TRIBE example Divide the variables to tribes of size lognloglogn+loglog 2 f=1 iff for some tribe all variables equal to 1 Ik(f) behaves like Klogn/n for some constant K. 13

KKL theorem: There always exists an influential variable Theorem (Kahn, Kalai, Linial 1988) Let f be a Boolean function and suppose that Prob(f=1)=s Then there is a variable k such that Ik(f) > C s (1 -s) log n / n This result was a conjecture by Ben-Or and Linial 14

KKL theorem 15

Fourier Given a Boolean function f : {-1, 1}n {-1, 1}, the Fourier expansion of f is simply writing f(x) as a sum of multilinear (square free) monomials. 16

Fourier Spectrum For every set S of variables we have the associated Fourier coefficient. The sum of squares of Fourier coefficients is 1. This defines a probability distribution called the “Fourier spectrum” (or “Fourier distribution”). The probability that k belongs to S, when S is distributed according to the Fourier spectrum is the influence of variable k on f. 17

Fourier The study of Boolean functions based on their Fourier expansion is fruitful. It can be regarded as a very special case of spectral methods in graph theory. 18

Hypercontractivity A very useful technical tool: The ratio between p-norms of low degree polynomials is bounded. (Khintchine , Nelson, Bonami, Gross, Beckner… ) 19

A glance at further advances and open problems Extensions to general Bernoulli product spaces. Notions of influence for continuous product spaces; larger alphabets; graph products. Symmetry and influence Understanding influence of sets of variables; Power-laws for influence 20

Part II Choice, power, rationality and manipulation 21

Individual and collective rationality and judgments Boolean functions can model how individual preferences between two alternatives aggregate. We can consider aggregation of individual preferences between m alternatives. (Social welfare functions. ) We can consider aggregation of judgments on r different binary questions, when there are certain consistency requirements. (Judgment aggregation. ) 22

Elections: measures of power Influence= Banzhaf power index Shapley-Shubik power index: Integral over p of the influence of the kth player with respect to the Bernoulli probability with parameter p. (Sum to 1. ) 23

Voting Paradoxes Condorcet’s paradox: Arrow’s paradox: “Doctrinal paradox”: 24

Manipulation A social choice function is a function from the profile of individual order relations to the set of alternatives. Manipulation: reporting an incorrect preference relation will improve the outcome. 25

A measure for manipulation The Gibbard-Satterthwaite theorem asserts for a non dictatorial choice function with at least 3 possible outcomes there are preferences that leads to manipulation. The manipulation power (Friedgut, Kalai and Nisan) of an individual k for a social choice function f, denoted by Mk(f) is the probability that x’k is a profitable manipulation for voter k when the profile of preferences x 1 x 2 , …, xn and x’k are chosen uniformly at random. 26

Conclusion of the Algorithmic GT/Econ Part The notions discussed in this lecture (measures for influence, power, manipulation, noise sensitivity…) may be of interest to other GT/econ models. For example, the model of exchange economy. Off-topic comment: Why is it rational and important to give incentives to difficult technical works. Added in proof: There is a work supporting this thought 27 by Kleinberg and Oren.

Part III Randomness 28

Collective coin flipping We need to create a random bit using a protocol based on random bits contributed by n processors. Some of the processors are malicious. A simple suggestion: Choice based on a Boolean function where each processor contributes a single bit. (An often asked question: why not choose among these bits at random? ) 29

Randomness as a computational resource “So, yes, I know that theory folks consider derandomization an open problem, but from my perspective, it is a solved problem for all practical purposes. ” Anon. CSProf on Shtetl-Optimized 30

What is Randomness? What is randomness and what is probability are fundamental question in many areas. Computational complexity offers a deep understanding to randomness. Its asymptotic nature makes it of little use in statistics. Q: What allows much simpler answers in statistics? A: The interactive proof system known as ``statistical hypothesis testing''. 31

What is the source of randomness? Is human uncertainty the only source of randomness? What is the explanation of the apparent randomness of high-level phenomena in nature? For example the distribution of females vs. males in a population (I am referring to randomness in terms of the unpredictability and not in the sense of it necessarily having to be evenly distributed). 1. Is it accepted that these phenomena are not really random, meaning that given enough information one could predict it? If so isn't that the case for all random phenomena? 2. If there is true randomness and the outcome cannot be predicted - what is the origin of that randomness? (is it a result of the randomness in the micro world - quantum 32 phenomena etc. . . )

Sharp threshold phenomena: Determinism from randomness Law of large numbers: Large stochastic systems behave deterministically. Sharp threshold phenomena: Choose the value of the variables to be 1 with probability p, independently. The value of f will rapidly change from 0 to 1. 33

Sharp threshold phenomena: Influence is a sort of derivative. Large total influence corresponds to small threshold interval. Theorem (Friedgut Kalai, 96): Symmetry implies sharp threshold Theorem (Kalai, 04): Sharp threshold is equivalent to diminishing maximum Shapley. Shubik power index. The economics term: complete aggregation of information. 34

Influence without independence (Haggstrom, Kalai, Mossel; Graham, Grimmett. ) Influence version A: The probability that changing the value of a variable will change the value of the function. Influence version B: The (normalized) correlation between the value of the variable and the value of the function. 35

Part IV Noise 36

Noise Sensitivity We consider a BOOLEAN FUNCTION f : {-1, 1}n {-1, 1} f(x 1 , x 2, . . . , xn) Given x 1 , x 2, . . . , xn we define y 1 , y 2, . . . , yn as follows: xi = yi with probability 1 -t xi = -yi with probability t 37

Noise Sensitivity Let C(f; t) be the correlation between f(x 1 , x 2, . . . , xn) and f(y 1, y 2, . . . , yn) A sequence of Boolean function (fn ) is noise-sensitive if for every t>0, C(fn, t) tends to zero with n. 38

Noise Stability Lets fix a small s=0. 0001(say). A Boolean function is noise stable to noise level t if the probability that f(x 1 , x 2, . . . , xn) is different from f(y 1, y 2, . . . , yn) is smaller than s. 39

Noise sensitivity, and nonclassical stochastic processes; black noise Closely related notions to “noise sensitivity” were studied by Tsirelson and Vershik. In their terminology “noise sensitivity” translates to “non Fock processes”, “black noise”, and “non-classical stochastic processes”. Their motivation is closer to mathematical quantum physics. 40

BKS theorem Theorem (Benjamini, Kalai, Schramm 1999) Monotone balanced Boolean functions are noise sensitive unless they have substantial correlation with some weighted majority functions. 41

Percolation is Noise sensitive Corollary [BKS, 1999]: The crossing event for critical planar percolation model is noisesensitive Theorem (Schramm and Smirnov, 2010): Percolation is a 2 -dimensional black noise. 42

Percolation is Noise sensitive Imagine two separate pictures of n by n hexagonal models for percolation. A hexagon is grey with probability ½. If the grey and white hexagons are independent in the two pictures the probability for crossing in both is ¼. If for each hexagon the correlation between its colors in the two pictures is 0. 99, still the probability for crossing in both pictures is very close to ¼ as n grows! If you put one drawing on top of the other you will hardly 43 notice a difference!

Other cases of noise sensitivity First Passage Percolation (Benjamini, Kalai, Schramm) The recursive Majority on threes example by Ben-Or and Linial (BKS) Eigenvalues of random Gaussian matrices (Essentially follows from the work of Tracy-Widom) Here, we leave the Boolean setting. Examples related to random walks (required replacing the discrete cube by trees) and more. . . 44

Majority is noise stable Sheppard Theorem: (1899): Suppose that there is a probability t for a mistake in counting each vote. The probability that the outcome of the election are reversed is: arccos(1 t)/π +o(1) When t is small this behaves like t 1/2 45

Majority is noise stable (cont. ) Weighted majority functions are also noise stable (BKS, Peres) Is there a more stable voting rule? Sure! dictatorship 46

Majority is stablest! Theorem: Mossel, O’Donnell and Oleszkiewicz : (2005) Let (fn ) be a sequence of Boolean functions with diminishing maximum influence. I. e. , limn->∞ maxk Ik(fn) -> 0 Then the probability that the outcome of the election are reversed when for every vote there is a probability t it is flipped is at least (1 -o(1)) arccos(1 -t)/π 47

Majority is stablest: Two applications 1. The probabilities of cyclic outcomes for voting rules with diminishing influences are minimized for the majority voting rule. 2. Improving the Goemans-Williamson 0. 878567 approximation algorithm is hard, unique-game-hard. 48

Is the universe noise sensitive? Are the basic models of high energy physics noise stable? If this is indeed the case, does it reflect some law of physics? Otherwise, will noise sensitivity allow additional modeling power? 49

Part V: Computational complexity Are the notions discussed here related to computational complexity? (We already mentioned relation to PCP; there are some interesting connection to randomized decision trees. ) 50

Diversion: CC and modeling

Computational complexity and Influence (… Ajtai, Furst, Saxe, Sipser, Yao, Hastad, Boppana, Linial, Mansour, Nisan…) The total influence of Boolean functions that can be described by depth D size M Boolean circuits is at most log MD-1 52

Computational complexity and Noise stability Conjecture 1: Let f be a monotone Boolean function described by monotone threshold circuits of size M and depth D. Then f is stable to (1/t)-noise where t=log M 100 D Conjecture 2: For some η >0, every balanced monotone functions in TC 0 have correlation at least η with a function in monotone TC 0. 53

Complexity of sampling the Fourier spectrum Suppose that f is a Boolean function in P. Can we approximately sample according to its Fourier spectrum? This is unknown and it might be hard. But. . . It is in BQP. (Namely, it is known to be easy for quantum computers. ) 54

Computation with noise Fault tolerant (quantum) computation. Are quantum computers possible? (This is a main research interest for me in recent years. ) The hope regarding FTQC: No matter what the quantum computer computes or simulates, nearly all of the noise will be a mixture of states that are not codewords in the error correcting code, but which are correctable to states in the code. The concern: The process for creating a quantum error correcting code will necessarily lead to a mixture of the desired codeword with undesired codewords. 55

Polymath 3 A recent endeavor: you are most welcome to join Details can be found on my blog http: //gilkalai. wordpress. com/2010/02/10/noisestability-and-threshold-circuits/ 1) The AC 0 analog 2) Positivity Vs Monotonicity 3) Natural Proof Obstructions 56

谢谢 Thank you! 谢谢 ! תודה רבה It is great to be in Beijing! 57