Autograph Toward Automated, Distributed Worm Signature Detection Hyang-Ah Kim Brad Karp Carnegie Mellon University Intel Research & Carnegie Mellon University Usenix Security 2004
Internet Worm n Internet Worm ¡ Large costs due to lost productivity n ¡ ¡ n Code Red: $2. 6 billion, Slammer: $1 billion Vulnerabilities still plentiful Smarter, faster, and more malicious worms easily possible [Staniford et al. , 2002] Internet Worm Quarantine Techniques ¡ ¡ ¡ Destination port blocking Infected source host IP blocking Content-based blocking [Moore et al. , 2003] Usenix Security 2004 2
Content-based Blocking 05: 45: 31. 912454 90. 196. 22. 196. 1716 > 209. 78. 235. 128. 80: . 0: 1460(1460) ack 1 Signature for Code. Red II win 8760 (DF) 0 x 0000 4500 05 dc 84 af 4000 6 f 06 5315 5 ac 4 16 c 4 E. . . @. o. S. Z. . . 0 x 0010 d 14 e eb 80 06 b 4 0050 5 e 86 fe 57 440 b 7 c 3 b. N. . . P^. . WD. |; 0 x 0020 5010 2238 6 c 8 f 0000 4745 5420 2 f 64 6566 P. "8 l. . . GET. /def 0 x 0030 6175 6 c 74 2 e 69 6461 3 f 58 5858 ault. ida? XXXXXXX 0 x 0040 5858 5858 XXXXXXXX. . . 0 x 00 e 0 5858 5858 XXXXXXXX 0 x 00 f 0 5858 5858 XXXXXXXX 0 x 0100 5858 5858 XXXXXXXX 0 x 0110 5858 5825 7539 3025 XXXXX%u 9090% 0 x 0120 7536 3835 3825 7563 6264 3325 7537 3830 u 6858%ucbd 3%u 780 0 x 0130 3125 7539 3025 7536 3835 3825 7563 1%u 9090%u 6858%uc 0 x 0140 6264 3325 7537 3830 3125 7539 3025 bd 3%u 7801%u 9090% 0 x 0150 7536 3835 3825 7563 6264 3325 7537 3830 u 6858%ucbd 3%u 780 0 x 0160 3125 7539 3039 3025 7538 1%u 9090%u 8 0 x 0170 3139 3025 7530 3063 3325 7530 3030 3325 190%u 00 c 3%u 0003% 0 x 0180 7538 6230 3025 7535 3331 6225 7535 3366 u 8 b 00%u 531 b%u 53 f 0 x 0190 6625 7530 : A Payload Content String Specific To A Worm 3037 3825 7530 3025 7530 f%u 0078%u 0000%u 0 0 x 01 a 0 303 d 6120 4854 5450 2 f 31 2 e 30 0 d 0 a 436 f 0=a. HTTP/1. 0. . Co Signature . . . Usenix Security 2004 3
Content-based Blocking Signature for Code. Red II Internet Traffic Filtering X Our network § Can be used by Bro, Snort, Cisco’s NBAR, . . . Usenix Security 2004 4
Signature derivation is too slow n Current Signature Derivation Process ¡ New worm outbreak Report of anomalies from people via phone/email/newsgroup Worm trace is captured Manual analysis by security experts Signature generation Labor-intensive, Human-mediated ¡ ¡ Usenix Security 2004 5
Goal Automatically generate signatures of previously unknown Internet worms § § as quickly as possible as accurately as possible Usenix Security 2004 6
Our Work n We focus on TCP worms that propagate via scanning Actually, any transport ¡ in which spoofed sources cannot communicate successfully ¡ in which transport framing is known to monitor n Worm’s payloads share a common substring ¡ Vulnerability exploit part is not easily mutable Usenix Security 2004 7
Outline n n Problem and Motivation Automated Signature Detection ¡ ¡ ¡ n Distributed Signature Detection ¡ ¡ n n Desiderata Technique Evaluation Tattler Evaluation Related Work Conclusion Usenix Security 2004 8
Desiderata n Automation: Minimal manual intervention n Signature quality: Sensitive & specific ¡ ¡ Sensitive: match all worms low false negative rate Specific: match only worms low false positive rate n Timeliness: Early detection n Application neutrality ¡ Broad applicability Usenix Security 2004 9
Automated Signature Generation Internet Traffic Filtering Autograph Monitor Our network X Signature n n Step 1: Select suspicious flows using heuristics Step 2: Generate signature using contentprevalence analysis Usenix Security 2004 10
S 1: Suspicious Flow Selection Reduce the work by filtering out vast amount of innocuous flows n Heuristic: Flows from scanners are suspicious Focus on the successful flows from IPs who made unsuccessful connections to more than s destinations for last 24 hours Suitable heuristic for TCP worm that scans network ¡ n Suspicious Flow Pool ¡ ¡ Autograph (s=2) Holds reassembled, suspicious flows captured during the last Non-existent time period t Non-existent Triggers signature generation if there are more than flows This flow will be selected Usenix Security 2004 11
S 1: Suspicious Flow Selection Reduce the work by filtering out vast amount of innocuous flows n Heuristic: Flows from scanners are suspicious Focus on the successful flows from IPs who made unsuccessful connections to more than s destinations for last 24 hours Suitable heuristic for TCP worm that scans network ¡ n Suspicious Flow Pool ¡ ¡ Holds reassembled, suspicious flows captured during the last time period t Triggers signature generation if there are more than flows Usenix Security 2004 12
S 2: Signature Generation Use the most frequent byte sequences across suspicious flows as signatures All instances of a worm have a common byte pattern specific to the worm Rationales ¡ Worms propagate by duplicating themselves ¡ Worms propagate using vulnerability of a service How to find the most frequent byte sequences? Usenix Security 2004 13
Worm-specific Pattern Detection n Use the entire payloads ¡ Brittle to byte insertion, deletion, reordering Flow 1 XXXX 9025753930399025 YYYY Flow 2 XXXXXXX 9025753930399025 YYYYY Usenix Security 2004 14
Worm-specific Pattern Detection Partition flows into non-overlapping small blocks and count the number of occurrences n Fixed-length Partition ¡ Still brittle to byte insertion, deletion, reordering Flow 1 XXXX 9025753930399025 YYYY Flow 2 XXXXXXX 9025753930399025 YYYYY Usenix Security 2004 15
Worm-specific Pattern Detection n Content-based Payload Partitioning (COPP) ¡ ¡ ¡ Determine boundaries of block using LBFS style Partition if Rabin fingerprint of a sliding window matches Breakmark Content Blocks Configurable parameters: content block size (minimum, average, maximum), breakmark, sliding window Flow 1 XXXX 9025753930399025 YYYY Flow 2 XXXXXXX 9025753930399025 YYYYY Breakmark = last 8 bits of fingerprint (9025) Usenix Security 2004 16
Why Prevalence? < Prevalence Distribution in Suspicious Flow Pool> Nimda Code. Red 2 Nimda (16 different payloads) Web. DAV exploit Innocuous, misclassified n n Worm flows dominate in the suspicious flow pool Content-blocks from worms are highly ranked Usenix Security 2004 17
Select Most Frequent Content Block f 0 C f 1 C D G f 2 A B D f 3 A C E f 4 A B E f 5 A B D f 6 H I f 7 I H J f 8 G F I J J Usenix Security 2004 18
Select Most Frequent Content Block f 0 C F f 1 C D G f 2 A B D f 3 A C E E f 4 A B E B D f 5 f 6 A B D H I J f 7 I H J f 8 G I J C F f 1 C D G f 2 A B D f 3 A C f 4 A f 5 A A f 6 A B H D IJ J C I A B I D IJ J C H f 7 A B G D IJ J C I f 8 E G H F Usenix Security 2004 19
Select Most Frequent Content Block f 0 p≥ 3 I J A B C D I J E A B D H I J I H J f 8 G I J G H A B E f 7 C D A C E f 5 f 6 A B D f 4 A C D G f 3 W≥ 90% f 1 f 2 Signature: C F G H F Usenix Security 2004 20
Select Most Frequent Content Block f 0 p≥ 3 I J A B C D I J E A B D H I J I H J f 8 G I J G H A B E f 7 C D A C E f 5 f 6 A B D f 4 A C D G f 3 W≥ 90% f 1 f 2 Signature: A C F G H F Usenix Security 2004 21
Select Most Frequent Content Block f 0 p≥ 3 I J A B C D I J E A B D H I J I H J f 8 G I J G H A B E f 7 C D A C E f 5 f 6 A B D f 4 A C D G f 3 W≥ 90% f 1 f 2 Signature: A C F G H F Usenix Security 2004 22
Select Most Frequent Content Block f 0 p≥ 3 I J C A B D H I J I H J f 8 G I J G H I A B E f 7 J A C E f 5 f 6 I A B D f 4 W≥ 90% C D G f 3 I f 1 f 2 Signature: A C F G H D F Usenix Security 2004 23
Select Most Frequent Content Block f 0 A B D A C E f 4 A B E f 5 f 6 A B D H I J f 7 p≥ 3 C D G f 3 W≥ 90% f 1 f 2 Signature: A U I C F I H J f 8 G I J C C G D F Usenix Security 2004 24
Outline n n Problem and Motivation Automated Signature Detection ¡ ¡ ¡ n Distributed Signature Detection ¡ ¡ n n Desiderata Technique Evaluation Tattler Evaluation Related Work Conclusion Usenix Security 2004 25
Behavior of Signature Generation n Objectives ¡ n Metrics ¡ ¡ n Effect of COPP parameters on signature quality Sensitivity = # of true alarms / total # of worm flows false negatives Efficiency = # of true alarms / # of alarms false positives Trace ¡ ¡ Contains 24 -hour http traffic Includes 17 different types of worm payloads Usenix Security 2004 26
Signature Quality n n Larger block sizes generate more specific signatures A range of w (90 -95%, workload dependent) produces a good signature Usenix Security 2004 27
Outline n n Problem and Motivation Automated Signature Detection ¡ ¡ ¡ n Distributed Signature Detection ¡ ¡ n n Desiderata Technique Evaluation Tattler Evaluation Related Work Conclusion Usenix Security 2004 28
Problem: Slow Payload Accumulation n Before signature generation, ¡ ¡ n Detect scanners (possibly infected hosts) Aggressiveness (s ) of flow selection heuristics Accumulate payloads enough for content analysis Earliness ( ) of signature generation trigger Infected vulnerable hosts ( =5, 63 monitors) Info Sharing Autograph Monitor None Luckiest Average Scanners, Signatures Average Aggressiveness of flow selection s=1 s=4 2% 60% 25% -<1% Usenix Security 2004 15% 29
Faster Signature Detection Share the scanner information with others Network C Traffic Autograph Filtering Monitor Internet Network B n Our network tattler Network A Usenix Security 2004 30
Benefit from tattler n Objective ¡ n Measure the detection speed and the signature quality Methodology ¡ Trace generation n n ¡ n Background noise flows from the real 24 hr trace Captured worm flows from simulation (63 monitors) Signature generation with COPP varying suspect flow pool size Metrics ¡ ¡ Percentage of infected hosts Number of unspecific signatures (that causes false positives) Usenix Security 2004 31
Tradeoff: Speed vs. Quality x x Decreasing s and parameters § faster signature generation § more false positives Usenix Security 2004 = 15, s = 2, < 2% infected 32
Attacks n Overload due to flow reassembly Multiple instances of Autograph on separate HW (port-disjoint) Suspicious flow sampling under heavy load n Abuse Autograph for Do. S: pollute suspicious flow pool Port scan and then send innocuous traffic Distributed verification of signatures at many monitors ¡ Source-address-spoofed port scan Reply with SYN/ACK on behalf of non-existent hosts/services ¡ Usenix Security 2004 33
![Related Work n Early. Bird [Singh et al. 2003] Pure content-based approach first, then](https://present5.com/presentation/21231cee3bd1bdbc18cf04ec416d8833/image-34.jpg "Related Work n Early. Bird [Singh et al. 2003] Pure content-based approach first, then")
Related Work n Early. Bird [Singh et al. 2003] Pure content-based approach first, then address dispersion heuristic ¡ Targets a single high speed link; no flow reassembly Honey. Comb [Kreibich et al. 2003] ¡ Signature detection by Honeypot & LCS algorithm ¡ Targets host-based deployment; small number of flows Honeyd [Provos 2003] ¡ Use distributed honeypots to gather worm payloads & infected IP addresses quickly ¡ Focus on harvesting malicious traffic DOMINO [Yegneswaran et al. 2004] ¡ Scanner IP information sharing among distributed nodes ¡ Distributed monitoring assures earlier & more accurate detection ¡ n n n Usenix Security 2004 34
Future Work n n n Online evaluation with diverse traces Deployment on distributed sites Broader set of suspicious flow selection heuristics ¡ ¡ n Non-scanning worms (ex. hit-list worms, topological worms, email worms) UDP worms Distributed agreement for signature quality testing Usenix Security 2004 35
Conclusion n n Stopping spread of novel worms requires early generation of signatures Autograph: automated signature detection system ¡ ¡ ¡ n Suspicious flow selection→ Content prevalence analysis COPP: robustness against payload variability Distributed monitoring: faster signature generation Autograph finds sensitive & specific signatures early in real network traces Usenix Security 2004 36
Скачать презентацию Autograph Toward Automated Distributed Worm Signature Detection Hyang-Ah
21231cee3bd1bdbc18cf04ec416d8833.ppt