Evaluation Methods for Internet Security Technology EMIST NSF

Evaluation Methods for Internet Security Technology (EMIST) NSF Cyber Trust PI Meeting and DETER workshop Newport Beach, CA, Sept. 2005 1

EMIST TEAM n n n n n PSU: G. Kesidis**(PI), P. Liu†, P. Mc. Daniel, D. Miller UCD: K. Levitt (PI), F. Wu*, J. Rowe, C. -N. Chua ICSI: V. Paxson* (PI), N. Weaver* Purdue: S. Fahmy (PI), N. Shroff, E. Spafford SPARTA: D. Sterne (PI), S. Schwab*, R. Ostrenga, R. Thomas, S. Murphy, R. Mundy SRI: P. Porras, L. Breismeister **overall PI, *expt lead/co-lead, †EMIST ESVT lead PMs: Joe Evans (NSF) and Douglas Maughan (DHS) Sister project: DETER cyber security testbed 2

Outline n n n Team. Goals. Publications. Tools released. Talks for DETER workshop Wed 09/28/05. Y 3 activities. 3

EMIST goals n n n Develop scientifically rigorous testing frameworks and methodologies for defenses against attacks on network infrastructure: scale-down with fidelity. Develop experiments to yield deeper understanding of how previous attacks have, and future attacks will, affect the Internet and its users. Develop prototypical experiments (benchmarks) and associated databases of: n n n topologies and topology generators attack and background traffic traces and generators defenses special-purpose devices (meters, virtual nodes, etc. ) metrics for scale-down fidelity, performance, overhead, etc. 4

EMIST goals (cont) n n n Consult in the build-out of the DETER testbed and demonstrate its usefulness to vendors, researchers and customers of defense technology. Allow for open, convenient, rigorous, unbiased and secure testing of cyber defenses on DETER in order to expedite their commercial deployment. Quickly and publicly disseminate our results. 5

2004 EMIST publications n n n N. Weaver, I. Hamadeh, G. Kesidis and V. Paxson, “Preliminary results using scale-down to explore worm dynamics”, in Proc. ACM WORM, Washington, DC, Oct. 29, 2004. P. Porras, L. Biesemeister, K. Levitt, J. Rowe, K. Skinner, A. Ting, “A hybrid quarantine defense”, in Proc. ACM WORM, Washington, DC, Oct. 29, 2004. S. T. Teoh, K. Zhang, S. -M. Tseng, K. -L. Ma and S. F. Wu, “Combining visual and automated data mining for near-real-time anomaly detection and analysis in BGP”, in Proc. ACM Viz. SEC/CMSEC-04, Washington, DC, Oct. 29, 2004. 6

2005 EMIST publications n n n n n A. Kumar, N. Weaver and V. Paxson, "Exploiting Underlying Structure for Detailed Reconstruction of an Internet-scale Event", in Proc. ACM IMC 2005. R. Pang, M. Allman, M. Bennett, J. Lee, V. Paxson, B. Tierney, "A First Look at Modern Enterprise Traffic ", in Proc. ACM IMC 2005. S. Schwab, B. Wilson, R. Thomas, “Methodologies and Metrics for the Testing and Analysis of Distributed Denial of Service Attacks and Defenses, ” MILCOM, Atlantic City, NJ, Oct. 2005. L. Li, S. Jiwasurat, P. Liu, G. Kesidis, Emulation of Single Packet UDP Scanning Worms in Large Enterprises, In Proc. 19 International Teletraffic Congress (ITC-19), Beijing, Aug. 2005. Q. Gu, P. Liu, C. -H. Chu, Hacking Techniques in Wired Networks, In The Handbook of Information Security , Hossein Bidgoli et al. (eds. ), John Wiley & Sons. S. Sellke, N. B. Shroff, and S. Bagchi, "Modeling and Automated Containment of Worms", In Proceedings of the International Conference in Dependable Systems and Networks (DSN), June 2005. R. Chertov, S. Fahmy, and N. B. Shroff, "Emulation versus Simulation: A Case Study of TCP-Targeted Denial of Service Attacks", Purdue University Technical Report, September 2005. L. Briesemeister and P. Porras. Microscopic simulation of a group defense strategy. In Proceedings of Workshop on Principles of Advanced and Distributed Simulation (PADS) , pages 254 -261, June 2005. C. H. Tseng, T. Song, P. Balasubramanyam, C. Ko, and K. Levitt, "A Specification-based Intrusion Detection Model for OLSR“, in Proc. RAID, Sept. 2005. 7

2005 EMIST publications n n n K. Zhang, S. Teoh, S. Tseng, R. Limprasittipom, C. Chuah, K. Ma, and S. F. Wu. PERFORMING BGP EXPERIMENTS ON A SEMI-RELISTIC INTERNET TESTBED ENVIRONMENT. in the 2 nd International Workshop on Security in Distributed Systems (SDCS), conjunction with ICDCS, 2005. W. Huang, J. Cong, C. Wu, F. Zhao, and S. F. Wu. DESIGN, IMPLEMENTATION, AND EVALUATION OF FRITRACE. in 20 th IFIP International Information Security Conference, May, 2005, Chiba, Japan, Kluwer Academic Publishers. G. Hong, F. Wong, S. F. Wu, B. Lilja, T. Y. Jansson, H. Johnson, and A. Nilsson. TCPTRANSFORM: PROPERTY-ORIENTED TCP TRAFFIC TRANSFORMATION. in GI/IEEE SIG SIDAR Conference on Detection of Intrusions and Malware & Vulnerability Assessment (DIMVA), Vienna, Austria, July, 2005, LNCS, Springer. J. Crandall, S. F. Wu, and F. Chong. EXPERIENCES USING MINOS AS A TOOL FOR CAPTURING AND ANALYZING NOVEL WORMS FOR UNKNOWN VULNERABILITIES. in GI/IEEE SIG SIDAR Conference on Detection of Intrusions and Malware & Vulnerability Assessment (DIMVA), Vienna, Austria, July, 2005, LNCS, Springer. G. H. Hong and S. F. Wu. ON INTERACTIVE INTERNET TRAFFIC REPLAY. in the 8 th Symposium on Recent Advanced Intrusion Detection (RAID), Seattle, September, 2005, LNCS, Springer. J. Crandall, Z. Su, S. F. Wu, and F. Chong. ON DERIVING UNKNOWN VULNERABILITIES FROM ZERO-DAY POLYMORPHIC & METAMORPHIC WORM EXPLOITS. To appear in 12 th ACM Conference on Computer & Communication Security (CCS’ 2005), Alexandria, November 7 -11, 2005. 8

EMIST tools n EMIST Experiment Specification and Visualization Tool (ESVT) 2. 0 released in May ’ 05 with: n n n n more advanced traffic viz features including link data and SQL interface, and ability to import output from a scale-free topology generator (with associated plotting tool). Offline netflow audit tool released in May ’ 05. Online Scriptable Event System (SES) and, data analysis measurement tools. XML worm configuration and worm modeling. TCPOpera traffic generator and ELISHA viz tool. BGP topology capture tool. Experimental technical reports. 9

ICSI worm demo: source models for testing net-based detectors n n We are developing layer 4 (TCP/UDP) “source models”. Process of representing normal systems: n n n Near-term goal is to mimic the Layer 4 behavior of normal hosts Testing against Approximate TRW worm containment n n Derived from traces of a medium-scale enterprise (10 K hosts) Store traffic information in database Classify host types & application sessions based on measurements Create background traffic by sampling hosts and sessions Overlay worm traffic by adding worm-functionality to models Longer term goals: n n investigate *abstract* source models apply to other containment technology 10

UC Davis / SRI worm demo: collaborative host-based defense n n Hosts that are not protected by network defenses can protect themselves from worm attack by collaborating with collections of other hosts to exchange alerts. A preliminary end-host collaborative worm defense exchanging failed connection reports will be demonstrated: n n n with respect to its ability to protect against worm spread in the presence of realistic background traffic. A 2000 virtual node experiment that uses our two tools: n n the NTGC traffic generator and the UCD Worm Emulator 11

SPARTA DDo. S demo n n n Flood. Watch defense deployed on both PCs and Cloud. Shield appliances, as well as Juniper routers. A range of data collection and EVST visualization tools will be explored. The theme is examination of the experimental methodology, in particular: n n the degree to which accurate detection and response characteristics can be calculated versus the limited fidelity of generated background traffic. 12

Purdue: Method and Tools for High. Fidelity Emulation of Do. S Attacks n n n Simulation versus emulation of Do. S attack experiments are compared. As a case study, we considered low-rate TCP-targeted Do. S attacks. Specific measurement-fidelity issues of the DETER testbed were resolved. We found that software routers such as Click provide a flexible experimental platform, but require detailed understanding of the underlying network device drivers to ensure they are correctly used. We also found that an analytical model and ns-2 simulations closely match with typical values of attack pulse lengths and router buffer sizes. 13

UCD: Requirements and Tools for Routing Experiments n Tools: Requirements and Design (with SPARTA) n n ER (Entity Relationship) Information Visualization Experiments: n Interaction of BGP/OSPF/P 2 P n n Per-Update OASC Experiment n n Cross-layer routing dynamics/interactions Analysis of address ownership DDo. S/Routing Interaction (with Purdue) n DDo. S impacts on BGP 14

PSU BGP demo: Large-Scale e. BGP Simulator (LSEB) n n Our goal is large Internet-scale (global) routing attack modeling and measurement. Methodology: n n n intial AS topologies drawn from PREDICT Routeviews 20 k java threads running across DETER hosts simulate all BGP message level interactions maintain route tables for all reachable prefixes Future work: n n realistic AS forwarding delay models modeling i. BGP scale-down of experiments with more complex/realistic BGP speakers defense deployment and evaluation on DETER 15

PSU ESVT demo n n ESVT rendering of UDP/TCP worm emulation in an enterprise: We have emulated SQL slammer on a 1000 node enterprise network and compared the realism achieved by VM (jail), real LANs, and virtual nodes. We are currently emulating TCP Blaster worm considering issues including the fidelity of our Blaster modeling technique, and the impact of background traffic. Note that no defense is involved, just a local block of dark addresses used for detection. 16

Y 3 Activities n Release of reusable code developed for on-going attack/defense experiments, in particular: n n n ESVT 3. 0+ with integrated trace audit tool, spectral analysis, etc. Synthesize background traffic analogous to trace data in DETER experiments on same topology. BGP ESVT. Continued outreach, in particular BGP ESVT components to the ops community. Collaborate with DETER on, e. g. , experimental workbench (SEW), RIB output collection. 17

Y 3 Activities (cont) n For each attack experiment, a summary document that described in particular: n n Experimental Tech Reports: n n n Experimental methodologies. Metrics for experimental realism in defense evaluation. Benchmark attack experiments for specific classes of defenses. Experiment archiving and repeatability issues. Critical assessments of all items in deterlab’s experimenters’ tools web pages. Summer 2006 attack/defense demonstration experiments. 18