Network-based Intrusion Detection, Prevention and Forensics System Yan Chen Department of Electrical Engineering and Computer Science Northwestern University Lab for Internet & Security Technology (LIST) http: //list. cs. northwestern. edu 1
The Spread of Sapphire/Slammer Worms 2
Current Intrusion Detection Systems (IDS) • Mostly host-based and not scalable to highspeed networks – Slammer worm infected 75, 000 machines in <10 mins – Host-based schemes inefficient and user dependent » Have to install IDS on all user machines ! • Mostly simple signature-based – Cannot recognize unknown anomalies/intrusions – New viruses/worms, polymorphism 3
Current Intrusion Detection Systems (II) • Cannot provide quality info forensics or situational-aware analysis – Hard to differentiate malicious events with unintentional anomalies » Anomalies can be caused by network element faults, e. g. , router misconfiguration, link failures, etc. , or application (such as P 2 P) misconfiguration – Cannot tell the situational-aware info: attack scope/target/strategy, attacker (botnet) size, etc. 4
Network-based Intrusion Detection, Prevention, and Forensics System • Online traffic recording [SIGCOMM IMC 2004, INFOCOM 2006, To. N to appear] – Reversible sketch for data streaming computation – Record millions of flows (GB traffic) in a few hundred KB – Small # of memory access per packet – Scalable to large key space size (232 or 264) • Online sketch-based flow-level anomaly detection [IEEE ICDCS 2006] [IEEE CG&A, Security Visualization 06] – Adaptively learn the traffic pattern changes – As a first step, detect TCP SYN flooding, horizontal and vertical scans even when mixed • Online stealthy spreader (botnet scan) detection [IWQo. S 2007] 5
Network-based Intrusion Detection, Prevention, and Forensics System (II) • Polymorphic worm signature generation & detection [IEEE Symposium on Security and Privacy 2006] [IEEE ICNP 2007 to appear] • Accurate network diagnostics [ACM SIGCOMM 2006] [IEEE INFOCOM 2007] • Scalable distributed intrusion alert fusion w/ DHT [SIGCOMM Workshop on Large Scale Attack Defense 2006] • Large-scale botnet and P 2 P misconfiguration event forensics [work in progress] 6
System Deployment • Attached to a router/switch as a black box • Edge network detection particularly powerful LAN Switch Router Splitter Switch Inter net Splitter scan port Router Switch LAN HPNAIDM system (b) Original configuration LAN Switch LAN (a) RAND system Router Switch LAN Inter net RAND system Inter net LAN scan port RAND system Monitor each port separately (c) Monitor aggregated 7 traffic from all ports
Hamsa: Fast Signature Generation for Zero-day Polymorphic Worms with Provable Attack Resilience Zhichun Li, Manan Sanghi, Yan Chen, Ming-Yang Kao and Brian Chavez Northwestern University
Desired Requirements for Polymorphic Worm Signature Generation • Network-based signature generation – Worms spread in exponential speed, to detect them in their early stage is very crucial… However » At their early stage there are limited worm samples. – The high speed network router may see more worm samples… But » Need to keep up with the network speed ! » Only can use network level information
Desired Requirements for Polymorphic Worm Signature Generation • Noise tolerant – Most network flow classifiers suffer false positives. – Even host based approaches can be injected with noise. • Attack resilience – Attackers always try to evade the detection systems • Efficient signature matching for high-speed links No existing work satisfies these requirements !
Outline • • • Motivation Hamsa Design Model-based Signature Generation Evaluation Related Work Conclusion
Choice of Signatures • Two classes of signatures – Content based » Token: a substring with reasonable coverage to the suspicious traffic » Signatures: conjunction of tokens – Behavior/vulnerability based • Our choice: content based – Fast signature matching. ASIC based approach can archive 6 ~ 8 Gb/s – Generic, independent of any protocol or server
Unique Invariants of Worms • Protocol Frame Invariants – The code path to the vulnerability part, usually infrequently used – Code-Red II: ‘. ida? ’ or ‘. idq? ’ • Control Data: leading to control flow hijacking – Hard coded value to overwrite a jump target or a function call • Worm Executable Payload – CLET polymorphic engine: ‘ 0x 8 b’, ‘xffxff’ and ‘tx 07xeb’ • Possible to have worms with no such invariants, but very hard
Hamsa Architecture
Hamsa Design • Key idea: model the uniqueness of worm invariants – Greedy algorithm for finding token conjunction signatures • Highly accurate while much faster – Both analytically and experimentally – Compared with the latest work, polygraph – Suffix array based token extraction • Provable attack resilience guarantee • Noise tolerant
Hamsa Signature Generator • Core part: Model-based Greedy Signature Generation • Iterative approach for multiple worms
Outline • • • Motivation Hamsa Design Model-based Signature Generation Evaluation Related Work Conclusion
Problem Formulation Maximize the coverage in the suspicious pool Signature Generator Normal pool false positive bound With noise NP-Hard! r Signature False positive in the normal pool is bounded by r
Model Uniqueness of Invariants t 1 U(1)=upper bound of FP(t 1) FP 21% t 2 U(2)=upper bound of FP(t 1, t 2) Joint FP with t 1 2% 9% 0. 5% 17% 1% 5% The total number of tokens bounded by k*
Signature Generation Algorithm token extraction Suspicious pool FP) (COV, (82%, 50%) (70%, 11%) (67%, 30%) (62%, 15%) (50%, 25%) (41%, 55%) (36%, 41%) (12%, 9%) Order by coverage t 1 u(1)=15% tokens
Signature Generation Algorithm Signature t 1 (COV, FP) (82%, 50%) (69%, 9. 8%) (70%, 11%) (68%, 8. 5%) (67%, 30%) (67%, 1%) (62%, 15%) (40%, 2. 5%) (50%, 25%) (35%, 12%) (41%, 55%) (31%, 9%) (36%, 41%) (10%, 0. 5%) (12%, 9%) Order by joint coverage with t 1 t 2 u(2)=7. 5%
Algorithm Analysis • Runtime analysis O(T*(|M|+|N|)) • Provable Attack Resilience Guarantee – Analytically bound the worst attackers can do! – Example: K*=5, u(1)=0. 2, u(2)=0. 08, u(3)=0. 04, u(4)=0. 02, u(5)=0. 01 and r=0. 01 Noise ratio 5% 10% 20% FP upper bound 1% 1% 1% FN upper bound 1. 84% 3. 89% 8. 75% – The better the flow classifier, the lower are the false negatives
Attack Resilience Assumptions • • • – – – • Two common assumptions for any sig generation sys Two unique assumptions for token-based schemes Attacks to the flow classifier Our approach does not depend on perfect flow classifiers With 99% noise, no approach can work! High noise injection makes the worm propagate less efficiently. Enhance flow classifiers
Improvements to the Basic Approach • Generalizing Signature Generation – • use scoring function to evaluate the goodness of signature Iteratively use single worm detector to detect multiple worms – – At the first iteration, the algorithm find the signature for the most popular worms in the suspicious pool. All other worms and normal traffic treat as noise.
Outline • • • Motivation Hamsa Design Model-based Signature Generation Evaluation Related Work Conclusion
Experiment Methodology • Experiential setup: – Suspicious pool: » » – • Three pseudo polymorphic worms based on real exploits (Code -Red II, Apache-Knacker and ATPhttpd), Two polymorphic engines from Internet (CLET and TAPi. ON). Normal pool: 2 hour departmental http trace (326 MB) Signature evaluation: – – False negative: 5000 generated worm samples per worm False positive: » » » 4 -day departmental http trace (12. 6 GB) 3. 7 GB web crawling including. mp 3, . rm, . ppt, . pdf, . swf etc. /usr/bin of Linux Fedora Core 4
Results on Signature Quality Training FN Evaluation Binary Worms FN FP evaluation FP Signature Code-Red II 0 0 0 {'. ida? ': 1, '%u 780': 1, ' HTTP/1. 0rn': 1, 'GET /': 1, '%u': 2} CLET 0 0. 109% 0 0. 06236% 0. 268% {'0x 8 b': 1, 'xffxff': 1, 'tx 07xeb': 1} • Single worm with noise – – • Training FP Suspicious pool size: 100 and 200 samples Noise ratio: 0%, 10%, 30%, 50%, 70% Noise samples randomly picked from the normal pool Always get above signatures and accuracy. Multiple worms with noises give similar results
Speed Results • • – – – Implementation with C++/Python 500 samples with 20% noise, 100 MB normal traffic pool, 15 seconds on an XEON 2. 8 Ghz, 112 MB memory consumption Speed comparison with Polygraph Asymptotic runtime: O(T) vs. O(|M|2), when |M| increase, T won’t increase as fast as |M|! Experimental: 64 to 361 times faster (polygraph vs. ours, both in python)
Outline • • • Motivation Hamsa Design Model-based Signature Generation Evaluation Related Work Conclusion
Related works Hamsa Polygraph CFG PADS Nemean COVERS Malware Detection Host Network or host based Network Host Content or behavior based Content based Behavior Content based Behavior based Noise tolerance Yes (slow) Yes No No Yes Multi worms Yes in one protocol Yes (slow) Yes No Yes Yes On-line sig matching Fast Slow Fast Slow Generality General purpose Protocol specific Server specific General purpose Provable atk Yes resilience No No No Information egp exploited egp p egp e eg p
Conclusion • Network based signature generation and matching are important and challenging Hamsa: automated signature generation • – – • Fast Noise tolerant Provable attack resilience Capable of detecting multiple worms in a single application protocol Proposed a model to describe the worm invariants
Outline • Overview of Network Intrusion Detection, Prevention and Forensics System • Case Study: Polymorphic Worm Signature Generation • Student recruiting
Northwestern Lab for Internet and Security Technology (LIST) • About Northwestern Univ. – US News and World Report, overall ranking #14, the Engineering grad school ranking #21. – On the Michigan lake, close to Chicago downtown • Sponsors for LIST: – Department of Energy (Early CAREER Award) – Air Force Office of Scientific Research (Young Investigator Award) – National Science Foundation – Microsoft Research – Motorola Inc.
Recruiting Ph. D. Students • Bachelor in Computer Science or Computer Engineering • Research experience a big plus • TOEFL • GRE • Strongly motivated in independent research • Feel free to talk to me after the talk 34
Скачать презентацию Network-based Intrusion Detection Prevention and Forensics System Yan
e46dcc7851b8067a10366fbb39049112.ppt