CS 151 Complexity Theory Lecture 3 April 10

CS 151 Complexity Theory Lecture 3 April 10, 2017

Padding and succinctness Two consequences of measuring running time as function of input length: • “padding” – suppose L EXP, and define N : x L, N = 2|x|k } PADL = { x# – TM that decides PADL: ensure suffix of N #s, ignore #s, then simulate TM that decides L – running time now polynomial ! April 10, 2017 2

Padding and succinctness • converse (intuition only): “succinctness” – suppose L is P-complete – intuitively, some inputs are “hard” -- require full power of P – SUCCINCTL has inputs encoded in different form than L, some exponentially shorter – if “hard” inputs are exponentially shorter, then candidate to be EXP-complete April 10, 2017 3

Succinct encodings • succinct encoding for a directed graph G= (V = {1, 2, 3, …}, E): • a succinct encoding for a variable-free Boolean circuit: i 1 iff wire from gate i to gate j j type of gate i i April 10, 2017 1 iff (i, j) E j type of gate j 4

An EXP-complete problem • Succinct Circuit Value: given a succinctly encoded variable-free Boolean circuit (gates , , , 0, 1), does it output 1? Theorem: Succinct Circuit Value is EXPcomplete. • Proof: – in EXP (why? ) – L arbitrary language in EXP, TM M decides L nk steps in 2 April 10, 2017 5

An EXP-complete problem – tableau for input x = x 1 x 2 x 3…xn: x_____ height, nk width 2 – Circuit C from CVAL reduction has size 2 nk). O(2 – TM M accepts input x iff circuit outputs 1 April 10, 2017 6

An EXP-complete problem – Can encode C succinctly: 1 iff wire from gate i to gate j type of gate i type of gate j i j • if i, j within single STEP circuit, easy to compute output • if i, j between two STEP circuits, easy to compute output • if one of i, j refers to input gates, consult x to compute output April 10, 2017 7

Summary • Remaining TM details: big-oh necessary. • First complexity classes: L, P, PSPACE, EXP • First separations (via simulation and diagonalization): P ≠ EXP, L ≠ PSPACE • First major open questions: ? ? L=P P = PSPACE • First complete problems: – CVAL is P-complete – Succinct CVAL is EXP-complete April 10, 2017 8

Summary EXP PSPACE P L April 10, 2017 9

Nondeterminism: introduction A motivating question: • Can computers replace mathematicians? L = { (x, 1 k) : statement x has a proof of length at most k } April 10, 2017 10

Nondeterminism: introduction • Outline: – nondeterminism – nondeterministic time classes – NP, NP-completeness, P vs. NP – co. NP – NTIME Hierarchy – Ladner’s Theorem April 10, 2017 11

Nondeterminism • Recall deterministic TM – Q finite set of states – ∑ alphabet including blank: “_” – qstart, qaccept, qreject in Q – transition function: δ : Q x ∑ ! Q x ∑ x {L, R, -} April 10, 2017 12

Nondeterminism • nondeterministic Turing Machine: – Q finite set of states – ∑ alphabet including blank: “_” – qstart, qaccept, qreject in Q – transition relation ∆ (Q x ∑) x (Q x ∑ x {L, R, -}) • given current state and symbol scanned, several choices of what to do next. April 10, 2017 13

Nondeterminism • deterministic TM: given current configuration, unique next configuration qstartx 1 x 2 x 3…xn x L qaccept qreject • nondeterministic TM: given current configuration, several possible next configurations April 10, 2017 14

Nondeterminism qstartx 1 x 2 x 3…xn x L “computation path” “guess” qaccept qreject • asymmetric accept/reject criterion April 10, 2017 15

Nondeterminism • all paths terminate • time used: maximum length of paths from root • space used: maximum # of work tape squares touched on any path from root April 10, 2017 16

Nondeterminism • NTIME(f(n)) = languages decidable by a multi-tape NTM that runs for at most f(n) steps on any computation path, where n is the input length, and f : N ! N • NSPACE(f(n)) = languages decidable by a multi-tape NTM that touches at most f(n) squares of its work tapes along any computation path, where n is the input length, and f : N ! N April 10, 2017 17

Nondeterminism • Focus on time classes first: NP = k NTIME(nk) NEXP = k April 10, 2017 n k) NTIME(2 18

Poly-time verifiers “witness” or Very useful alternate definition of NP: “certificate” efficiently Theorem: language L is in NP if and only if it verifiable is expressible as: L = { x| 9 y, |y| ≤ |x|k, (x, y) R } where R is a language in P. • poly-time TM MR deciding R is a “verifier” April 10, 2017 19

Poly-time verifiers • Example: 3 SAT expressible as 3 SAT = {φ : φ is a 3 -CNF formula for which assignment A for which (φ, A) R} R = {(φ, A) : A is a sat. assign. for φ} – satisfying assignment A is a “witness” of the satisfiability of φ (it “certifies” satisfiability of φ) – R is decidable in poly-time April 10, 2017 20

Poly-time verifiers L = { x | 9 y, |y| ≤ |x|k, (x, y) R } Proof: ( ) give poly-time NTM deciding L phase 1: “guess” y with |x|k nondeterministic steps phase 2: decide if (x, y) R April 10, 2017 21

Poly-time verifiers Proof: ( ) given L NP, describe L as: L = { x | 9 y, |y| ≤ |x|k, (x, y) R } – L is decided by NTM M running in time nk – define the language R = { (x, y) : y is an accepting computation history of M on input x} – check: accepting history has length ≤ |x|k – check: R is decidable in polynomial time – check: M accepts x iff y, |y| ≤ |x|k, (x, y) R April 10, 2017 22

Why NP? problem object we requirements are seeking • not a realistic model of computation • but, captures important computational feature of many problems: exhaustive search works efficient test: • contains huge number of natural, practical does y meet requirements? problems • many problems have form: L = { x | 9 y s. t. (x, y) 2 R} April 10, 2017 23

Why NP? • Why not EXP? – too strong! – important problems not complete. April 10, 2017 24

Relationships between classes • Easy: P µ NP, EXP µ NEXP – TM special case of NTM • Recall: L NP iff expressible as L = { x | 9 y, |y| · |x|k s. t. (x, y) 2 R} • NP µ PSPACE (try all possible y) • The central question: ? P = NP finding a solution vs. recognizing a solution April 10, 2017 25

NP-completeness • Circuit SAT: given a Boolean circuit (gates , , ), with variables y 1, y 2, …, ym is there some assignment that makes it output 1? Theorem: Circuit SAT is NP-complete. • Proof: – clearly in NP April 10, 2017 26

NP-completeness – Given L NP of form L = { x | 9 y s. t. (x, y) 2 R} x 1 x 2 … 1 iff (x, y) R xn y 1 y 2 … ym CVAL reduction for R – hardwire input x; leave y as variables April 10, 2017 27

NEXP-completeness • Succinct Circuit SAT: given a succinctly encoded Boolean circuit (gates , , ), with variables y 1, y 2, …, ym is there some assignment that makes it output 1? Theorem: Succinct Circuit SAT is NEXPcomplete. • Proof: – same trick as for Succinct CVAL EXPcomplete. April 10, 2017 28

Complement classes • In general, if C is a complexity class • co-C is the complement class, containing all complements of languages in C – L C implies ( * - L) co-C – ( * - L) C implies L co-C • Some classes closed under complement: – e. g. co-P = P April 10, 2017 29

co. NP • Is NP closed under complement? Can we transform this machine: x L qreject April 10, 2017 x L qaccept qreject into this machine? 30

co. NP • “proof system” interpretation: • Recall: L NP iff expressible as L = { x | 9 y, |y| ≤ |x|k, (x, y) R } “proof” “proof verifier” • languages in NP have “short proofs” • co. NP captures (in its complete problems) problems least likely to have “short proofs”. – e. g. , UNSAT is co. NP-complete April 10, 2017 31

co. NP • P = NP implies NP = co. NP • Belief: NP ≠ co. NP – another major open problem April 10, 2017 32

NTIME Hierarchy Theorem (Nondeterministic Time Hierarchy Theorem): For every proper complexity function f(n) ≥ n, and g(n) = ω(f(n+1)), NTIME(f(n)) ( NTIME(g(n)). April 10, 2017 33

NTIME Hierarchy Theorem Proof attempt : (what’s wrong? ) inputs Turing Machines Y n n Y n D: April 10, 2017 (M, x): does NTM M accept x in f(n) steps? n Y Y n Y 34

NTIME Hierarchy Theorem • Let t(n) be large enough so that can decide if NTM M running in time f(n) accepts 1 n, in time t(n). 1 n M 1 y n y ? ? ? . . . Mi-1 n y n ? ? ? Mi n y y ? ? ? D: April 10, 2017 1 t(n) ? . . . ? ? I’m responsible for dealing with NTM Mi . . . 35

NTIME Hierarchy Theorem • Enough time on input 1 t(n) to do the opposite of Mi(1 n): Mi D: April 10, 2017 1 n y ? ? ? . . . 1 t(n) ? . . . n 36

NTIME Hierarchy Theorem • For k in [n…t(n)] can to do same as Mi(1 k+1) on input 1 k Mi D: April 10, 2017 1 n y ? ? ? . . . 1 t(n) ? . . . n 37

NTIME Hierarchy Theorem • Did we diagonalize against Mi? – if L(Mi) = L(D) then: Mi 1 n y ? ? ? D: . . . 1 t(n) ? . . . n – equality along all arrows. – contradiction. April 10, 2017 38

NTIME Hierarchy Theorem • General scheme: – interval [1. . . t(1)] kills M 1 – interval [t(1)…t(t(1))] kills M 2 – interval [ti-1(1)…ti(1)] kills Mi • Running time of D on 1 n: f(n+1) + time to compute interval containing n • conclude D in NTIME(g(n)) (g(n) = ω(f(n+1))) April 10, 2017 39

Ladner’s Theorem • Assuming P ≠ NP, what does the world (inside NP) look like? NP: NPC P April 10, 2017 NP: P 40

Ladner’s Theorem (Ladner): If P ≠ NP, then there exists L NP that is neither in P nor NPcomplete. • Proof: “lazy diagonalization” – deal with similar problem as in NTIME Hierarchy proof April 10, 2017 41

Ladner’s Theorem • Can enumerate (TMs deciding) all languages in P. – enumerate TMs so that each machine appears infinitely often – add clock to Mi so that it runs in at most ni steps April 10, 2017 42

Ladner’s Theorem • Can enumerate (TMs deciding) all NPcomplete languages. – enumerate TMs fi computing all polynomialtime functions – machine Ni decides language SAT reduces to via fi if fi is reduction, else SAT (details omitted…) April 10, 2017 43

Ladner’s Theorem • Our goal: L NP that is neither in P nor NP complete April 10, 2017 Mi M 0 L N 0 Ni : : inputs : : 44

Ladner’s Theorem • Top half, assuming P ≠ NP: • focus on Mi Mi any x, can always find some z ≥ M 0 x on which SAT Mi and SAT differ (why? ) • for April 10, 2017 : : input x input z 45

Ladner’s Theorem • Bottom half, assuming P ≠ NP: TRIV = Σ* • for any x, can always find some z ≥ x on which Ni and TRIV differ (why? ) April 10, 2017 input z input x • focus on Ni TRIV N 0 Ni : : 46

Ladner’s Theorem • Try to “merge”: SAT TRIV • on input x, either – answer SAT(x) – answer TRIV(x) Mi M 0 L N 0 • if can decide which one in P, L NP Ni April 10, 2017 : : ≠ ≠ ≠ : . . . ≠ : 47

Ladner’s Theorem • General scheme: f(n) slowly increasing function L f(|x|) 0 0 0 1 1 1 2 2 . . . SAT . . . TRIV – f(|x|) even: answer SAT(x) – f(|x|) odd: answer TRIV(x) • notice choice only depends on length of input… that’s OK April 10, 2017 48

Ladner’s Theorem • 1 st attempt to define f(n) • “eager f(n)”: increase at 1 st opportunity • Inductive definition: f(0) = 0; f(n) = – if f(n-1) = 2 i, trying to kill Mi • if 9 z < 1 n s. t. Mi(z) ≠ SAT(z), then f(n) = f(n-1) + 1; else f(n) = f(n-1) – if f(n-1) = 2 i+1, trying to kill Ni • if 9 z < 1 n s. t. Ni(z) ≠ TRIV(z), then f(n) = f(n-1) + 1; else f(n) = f(n-1) April 10, 2017 49

Ladner’s Theorem • Problem: eager f(n) too difficult to compute • on input of length n, – look at all strings z of length < n – compute SAT(z) or Ni(z) for each ! • Solution: “lazy” f(n) – on input of length n, only run for 2 n steps – if enough time to see should increase (over f(n-1)), do it; else, stay same – (alternate proof: give explicit f(n) that grows slowly enough…) April 10, 2017 50

Ladner’s Theorem • Key: n eventually large enough to notice completed previous stage Mi L f . . . ≠ 0 0 1 1. . . k k k input z < x suppose k = 2 i April 10, 2017 input x • I’m supposed to ensure Mi is killed • I finally have enough time to check input z • I notice z did the job, increase f to k+1 51

Ladner’s Theorem • Inductive definition of f(n) – f(0) = 0 – f(n): for n steps compute f(0), f(1), f(2), … L f . . . 0 0 1 1. . . k k k. . . got this far in n steps April 10, 2017 input x, |x| = n 52

Ladner’s Theorem – if k = 2 i: • for n steps try (lex order) to find z s. t. SAT(z) ≠ Mi(z) and f(|z|) even • if found, f(n) = f(n-1)+1 else f(n-1) – if k = 2 i + 1: • for n steps try (lex order) to find z s. t. TRIV(z) ≠ Ni(z) and f(|z|) odd • if found, f(n) = f(n-1)+1 else f(n-1) April 10, 2017 53

Ladner’s Theorem • Finishing up: L = { x | x SAT if f(|x|) even, x TRIV if f(|x|) odd } • L NP since f(|x|) can be computed in O(n) time April 10, 2017 54

Ladner’s Theorem • suppose Mi decides L – f gets stuck at 2 i – L SAT for z : |z| > no – implies SAT P. Contradiction. • suppose Ni decides L – f gets stuck at 2 i+1 – L TRIV for z : |z| > no – implies L(Ni) P. Contradiction. • (last of diagonalization…) April 10, 2017 55