Analyzing Network Traffic in the Presence of Adversaries

Analyzing Network Traffic in the Presence of Adversaries Vern Paxson International Computer Science Institute / Lawrence Berkeley National Laboratory vern@icsi. berkeley. edu / vern@ee. lbl. gov October 18, 2004

Roadmap • In today’s Internet, attacks are the norm – Adversaries can create fundamental problems for network traffic analysis • #1 problem: evasion by ambiguity • “Active mapping” to resolve ambiguities • “Normalization” to eliminate ambiguities • #2: flooding directed at network devices • How to design robust analysis hardware

= 80% growth/year Data courtesy of Rick Adams

= 60% growth/year

= 596% growth/year

= 596% growth/year Courtesy Mark Dedlow

In Today’s Internet Attacks are the Norm Great interest in watching network traffic and analyzing what it’s doing • Watching: monitor traffic at chokepoints, capture copy or perhaps intercept • Analyzing: reconstruct protocol layers as seen by endpoints, interpret semantics • How hard can it be? • Attackers are adversaries: they don’t want to be caught and they want to make it painful for us to operate

The Problem of Evasion • Evasion raises fundamental problems • Network traffic seen from within a network is inherently ambiguous. • Analyzing network traffic at a high semantic level requires extensive state … … which an adversary can target. • Consider a network intrusion detection system (IDS; “Bro”) detecting occurrences of the string “root” inside a network connection (Let’s disregard the wholly separate issue of false positives: whether this is a good “signature”)

Detecting “root”: Attempt #1 • Method: scan each packet for ‘r’, ‘o’, ‘t’ – Perhaps using Boyer-Moore, Aho-Corasick, Bloom filters … 1 ……. …. root………. . ………… But: TCP protocol doesn’t preserve text boundaries 1 ……. …. ro 2 ot………. . …………

Detecting “root”: Attempt #2 • Method: remember match from end of previous packet 1 2 ……. …. ro + ot………. . ………… - Now we’re managing state But: TCP protocol doesn’t guarantee in-order arrival 2 ot………. . ………… ? ……. …. ro 1

Detecting “root”: Attempt #3 • Method: reassemble entire byte stream – Keep track of full TCP connection state – So much for “simple” – What happens if we run out of memory? • And: – Still evadable …

Evading Detection Via Ambiguous TCP Retransmission

Evading Detection Via Ambiguous TCP Retransmission

It’s Not Just TTL Expiration • Systematic study (w/ M. Handley & C. Kreibich) to analyze ambiguous protocol fields: – 73 exploitable ambiguities IP/TCP/UDP/ICMP – E. g: control flags, flow control window, “don’t fragment”, old timestamps, service class, redundant length field, filtering on unused bits – Internet protocols not designed for analysis – Attacker toolkits already exist for exploiting these • Answer: alert upon seeing ambiguous traffic?

The Problem of “Crud” • Unfortunately, ambiguities occur in benign traffic, too: – Legitimate tiny fragments, overlapping fragments – Receivers that acknowledge data they did not receive – Senders that retransmit different data than originally • In a diverse traffic stream, you will see these: – What is the intent? • Loss of alert precision “Maybe there’s an attack”

Countering Evasion-by-Ambiguity: Active Mapping • Idea (w/ Umesh Shankar, UCB): Probe end-host in advance to resolve vantagepoint ambiguities – E. g. , how many hops to it? – E. g. , how does it resolve ambiguous retransmissions? – Gray-box testing

Mapping Setup

Grey-box Inference of Reassembly Policy

A Plethora of Inferred Policies

Issues for Active Mapping • Probing for most ambiguities requires eliciting a response – Some hosts won’t respond when not actively engaged – For some responses, need to trick host into echoing back what it saw • Have to take churn into account – At a large site, something’s always changing – Lack of identity due to NAT, DHCP – Our implementation takes ≈ 5 sec/host

Countering Evasion-by-Ambiguity: Normalization • Idea (w/ Mark Handley, Christian Kreibich): Introduce network element to rewrite traffic passing through it to eliminate ambiguities – E. g. , regenerate low TTLs (dicey!) – E. g. , regularize flags, unused fields – E. g. , trim out-of-window data – E. g. , reassemble streams & remove inconsistent retransmissions

Issues for Normalization • Effect on end-to-end semantics? – Some normalizations harmless (e. g. , inconsistent streams) – Some actually improve protocol (e. g. , reliable RSTs) – Some degrade performance in the presence of cold start (e. g. , stripping TCP window scaling) • Performance: element is in-line – Prototype (1. 1 GHz): 400 Mbps – Would like to use custom hardware …

Robust Hardware for Analyzing Traffic in the Presence of Adversaries • Ongoing work w/ Sarang Dharmapurikar (WUSTL) • Basic building-block for boosting network analysis: in-line TCP stream reassembly – If data arrives in-sequence, hand it to analyzer module – If data arrives out-of-sequence, it creates a “hole” • Buffer for later delivery • How hard can it be?

How Much Buffer for Holes Do We Need? • Most previous work says: “Zero” – Skip out-of-sequence packets • Commercial work says: “Yes” – Claim out-of-sequence packets buffered, but with no details • Answer for sound operation depends critically on whether we consider adversaries …

Measured Buffer Required Per-Hole

Measured Duration of Holes

Instantaneous Aggregate Hole Buffer

How Much Buffer for Holes Do We Need? , con’t • Trace analysis says: a few hundred KB suffices even for a large site’s access link. . . • … But: an adversary can maliciously create holes, overflowing the buffer. On overflow, we can either: – Stop analyzing evicted connection, allowing adversary to evade – Kill unanalyzable connection, allowing adversary to inflict collateral damage

Adversary-Resistant Stream Reassembly • Trace analysis also says: – Very few connections have concurrent holes • Can limit adversary to one hole per connection – No hosts have concurrent connections w/ holes • Can limit adversary to one hole per Zombie • Consider randomized eviction: – If buffer size >> requirements of legit connections, then most evictions evict the attacker’s own holes

Zombie Equations • Let: – – – M, P = total memory (pages) available for holes Ml, Pl = memory (pages) for legitimate holes e = tolerable eviction rate for legit. connections r = rate at which a zombie can transmit (bytes/sec) g = page size (granularity) for hole buffer Z = # of zombies required to achieve eviction rate • Then for attacker creating small/large holes:

Zombie Implications • If we only terminate connections with > 2 packets buffered; allow each connection 10 KB of buffer; and use 512 MB DRAM … • … then collateral damage rate X of legitimate connections terminated per second is: By throwing memory at the problem, we can weather a large attack

Summary • The lay of the land has changed – Ecosystem of endemic hostility • Adversaries can exploit ambiguity and pressures of holding state to evade detection or inflict collateral damage • Internet protocols not designed with “wire analysis” in mind … • … But it is possible to design to address these issues if they are properly considered

Summary, con’t • Network analysis amidst adversaries is a new area: – Did not talk about: application-level evasion, polymorphism, tunneling, compromising passive monitors – In many ways, reminiscent of Internet measurement a decade ago: • Low-hanging fruit • Daunting problems • Fun!