Defending Distributed Systems Against Malicious Intrusions and Network Anomalies Kai Hwang Internet and Grid Computing Laboratory University of Southern California Keynote Presentation at the IEEE International Workshop on Security in Systems and Networks (SSN-2005), held in conjunction with the IEEE International Parallel and Distributed Processing Symposium (IPDPS-2005), Denver, Colorado, April 8, 2005 This presentation is based on research findings by USC Grid. Sec team. Project Web site: http: //Grid. Sec. usc. edu, supported by NSF ITR Grant No. 0325409, and contributed by Min Cai, Shanshan Song, Ricky Kwok, Ying Chen, and Hua Liu 1
Presentation Outline: § Security/privacy demands in networked or distributed computer systems § Grid. Sec Net. Shield architecture for defending distributed resource sites in Grids, clusters, etc. § Internet datamining for collaborative anomaly and intrusion detection system (CAIDS) with traffic episode rule training and analysis § Fast containment of internet worm outbreaks and tracking of related DDo. S attacks with distributedhashing overlays April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 2
Security and Privacy Demands in Network and Distributed Systems § § Trusted resource allocation, sharing, and scheduling § Intrusion and anomaly detection, attack repelling, trace back, pushback of attacks, etc § Fortification of hardware/software (firewalls, packet filters, VPN gateways, traffic monitors, security overlays, etc. ) § Self-defense toolkits/middleware for distributed defense, risk assessment, worm containment, response automation § Anonymity, confidentiality, data integrity, fine- grain access control, resolving conflicts in security policies, etc Secure communications among resource sites, clusters, and protected download among peer machines April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 3
Grid. Sec: A Grid Security ITR Project at USC Site S 1 Host 3 VPN Gateway 3 Host Internet 3 Host 2 3 Host 1 Site S 2 Host 3 VPN Gateway Host 3 Host Steps for automated self-defense at resource site : Step 1: Intrusion detected by host-based firewall /IDS Step 2: All VPN gateways are alerted with the intrusions Step 3: Gateways broadcast response commands to all hosts April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 4 Site S 3
The Net. Shield Architecture with Distributed Security Enforcement over a DHT Overlay April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 5
Building Encrypted Tunnels between Grid Resource Sites Through the DHT Overlay § The number of encrypted tunnels should grow with O(N) instead of O(N x N), where N is the number of Grid sites § Using shortest path, security policy is enforced with minimal VPN tunnels to satisfy special Grid requirements, automatically § How to integrate security policies from various private networks through the public network ? § How to resolve security policy conflicts among hosts, firewalls, switches, routers, and servers, etc. in a Grid environment ? April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 6
Trust Integration over a DHT Overlay V V Site S 3 Site S 2 Site S 1 Physical backbone Site S 4 DHT Overlay Ring V VPN Gateway Se. GO Server Trust Vector Trust vector propagation User application and Se. GO server negotiation Hosts Cooperating gateways working together to establish VPN tunnels for trust integration April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 7
USC Net. Shield Intrusion Defense System for Protecting Local Network of Grid Computing Resources ISP The Internet April 8, 2005, Kai Hwang Network Router The Net. Shield System Firewall Datamining for Anomaly Intrusion Detection (IDS) Risk Assessment System (RAS) http: //Grid. Sec. usc. edu Intrusion Response System (IRS) 8 Victim’s Internal Network
Alert Operations performed in local Grid sites and correlated globally April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 9
Basic Concept of Internet Episodes l Event Type: A, B, C, D, E, F, etc. l Event Sequence: e. g. , <(E, 31), (D, 32), (F, 33)> l Window: Event sequence with a particular width l Episode: partially ordered set of events, e. g. whenever A occurs, B will occur soon l Frequency of episode: fraction of windows in which episode occurs l Frequent episode: set of episodes having a frequency over a particular frequency threshold l Frequent episode rules are generated to describe the connection events April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 10
Frequent Episode Rules (FER) for Characterizing Network Traffic Connections E → D, F ( c, s ) The episode of 3 connection events (E, D, F) = (http, smtp, telnet). On the LHS , we have the earlier event E (http). On the RHS, we have two consequence events D (smtp) and F(telnet); where s is the support probablity and c is the confidence level specified below: (service = http, flag = SF) → (service = smtp, srcbyte = 5000), (service = telnet, flag = SF) (0. 8, 0. 9) Support probability s = 0. 9 and Confidence level c = 0. 8 that the episode will take place in a typical traffic stream April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 11
A Cooperative Anomaly and Intrusion Detection System (CAIDS), built with a Network Intrusion Detection System (NIDS) and an Anomaly Detection System (ADS) operating interactively through automated signature generation Training data from audit normal traffic records Single-connection attacks detected at packet level Audit records from traffic data IDS Known attack signatures from ISD provider April 8, 2005, Kai Hwang Signature Matching Engine Attack Signature Database Unknown or burst attacks New signatures from anomalies detected Episode Rule Database Episode Mining Engine Anomalies detected over multiple connections ADS Signature ADS http: //Grid. Sec. usc. edu Generator 12
Internet Datamining for Episode Rule Generation April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 13
Attack Spectrum from MIT Lincoln Lab in 10 Days of Experimentation April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 14
Automated Signature Generation from Frequent Episode Analysis 1. Label relevant connections to associate with an FER. Online traffic episode rules from the datamining engine Episode rules matching the normal FER database ? Yes Episode Frequency exceeding the rule threshold ? No Yes No (Massive attacks) 2 Calculate additional information such as connection count, average and percentage of connections, etc. 3 Select one of the predefined classifiers 4 Use the selected classifier to classify the attack class and find the relevant connections 5 Extract common features in all identified connections, such as the IP addresses, protocol, etc. to form the signature (Stealthy attacks) 2 Check error flags or other useful temporal statistics 3 Extract common features such as IP addresses, protocol, etc. to form the signature Adding new signatures to the Snort database Ignore the normal episode rules from legitimate users (No anomaly detected) April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 15
Successful Detection Rates of Snort , Anomaly Detection System (ADS), and the Collaborative Anomaly and Intrusion Detection System (CAIDS) April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 16
False Alarms out of 201 Attacks in CAIDS Triggered by Different Attack Types under Various Scanning Window Sizes Using larger windows result in more false alarms. Shorter windows in 300 sec or less are better in the sense that shorter episodes will be mined to produce shorter rules, leading to faster rule matching in the anomaly detection process April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 17
Detection Rates of Snort, ADS, and CAIDS under Various Attack Classes On the average, the CAIDS (white bars) outperforms the Snort and ADS by 51% and 40%, respectively April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 18
ROC Curves for 4 Attack Classes on The Simulated CAIDS April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 19
ROC Performance of Three Intrusion Detection Systems April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 20
Internet Worm and Flood Control: § A DHT-based Worm. Shield overlay network is under development at USC. § Fast worm signature generation and fast dissemination through both local and global address dispersion § Automated tracking of DDo. S attack-transit routers to cut off malicious packet flows for dynamic DDo. S flood control April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 21
The Worm. Shield Built with a DHT-based Overlay with Six Worm Monitors April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 22
The Worm. Shield Signature Generation Process April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 23
Signature Detection in Worm Spreading and the Growth of Infected hosts for Simulated Code. Red Worms on a Internet Configuration of 105, 246 Edge networks in 11, 342 Autonomous Systems Containing 338, 652 Vulnerable Hosts April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 24
Effects of Local Prevalence Threshold Worm spreading and the growth of infected hosts April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 25
Effects of Global Address Prevalence on Worm Spreading and the Growth of Infected Hosts April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 26
Reduction of Infected Hosts by Independent vs. Collaborative Monitoring over the Edge Networks April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 27
Packet/Flow Counting for Tracking Attack-Transit Routers (ATRs) April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 28
False Positive Rate of Identified ATRs April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 29
Other Hot Security Research Areas: l Efficient and enforceable trust models are very much in demand for networked and distributed systems: PKI services, VPN tunneling, trust negotiation, security overlays, reputation system etc. l Large-scale security benchmark experiments in open Internet environments are infeasible. The NSF/HSD DETER testbed should be fully used in performing such experiments to establish sustainable cybertrust over all edge networks. l Internet datamining for security control and for the guarantee of Quality-of-Service in real-life network applications – Interoperability between wired and wireless networks is a wide-open area for further research. April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 30
Final Remarks § The Net. Shield built with DHT-based security overlay networks support distributed intrusion and anomaly detection, alert correlation, collaborative worm containment, and flooding attack suppression. § The CAIDS can cope with both known and unknown network attacks, secure many cluster/Grid/P 2 P operations in using common Internet services: telnet, http, ftp, Email, SMTP, authentication, etc. § Automated virus or worm signature generation plays a vital role to monitory network epidemic outbreaks and to give early warning of large-scale system intrusions, network anomalies, and DDo. S flood attacks. Extensive benchmark experiments on the DETER test bed will prove the effectiveness. April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 31
Recent Related Papers: 1. M. Cai, K. Hwang, Y. K. Kwok, Y. Chen, and S. S. Song, “Fast Containment of Internet Worms and Tracking of DDo. S Attacks with Distributed-Hashing Overlays”, IEEE Security and Privacy, accepted to appear Nov/Dec. 2005. 2. K. Hwang, Y. Kwok, S. Song, M. Cai, R. Zhou, Yu. Chen, Ying. Chen, and X. Lou, “Grid. Sec: Trusted Grid Computing with Security Binding and Self-Defense against Network Worms and DDo. S Attacks”, International Workshop on Grid Computing Security and Resource Management (GSRM’ 05), in conjunction with ICCS 2005, Atlanta, May 22 -25, 2005. 3. M. Qin and K. Hwang, “Frequent Episode Rules for Internet Traffic Analysis and Anomaly Detection”, IEEE Network Computing and Application Symp. (NCA-2004), Cambridge, MA. August 31, 2004 4. K. Hwang, Y. Chen and H. Liu, “ Defending Distributed Computing Systems from Malicious Intrusions and Network Anomalies”, IEEE Workshop on Security in Systems and Networks (SSN’ 05), in conjunction with IEEE IPDPS 2005, Denver, April 8, 2005, Kai Hwang http: //Grid. Sec. usc. edu 32
Скачать презентацию Defending Distributed Systems Against Malicious Intrusions and Network
8dbeb9907df984240811fa3dec78d011.ppt