Malicious Code and Intruders Dr Ron Rymon Efi

Malicious Code and Intruders Dr. Ron Rymon Efi Arazi School of Computer Science IDC, Herzliya, 2007/8 Pre-requisite: Basic Cryptography, Authentication

Overview ¨ Malicious Code (Viruses) ¨ Intrusion Detection and Prevention ¨ Denial of Service

Malicious Code (Viruses) Main Sources: Stallings, F-Secure

Types of Malicious Code Need Host Program Trapdoors Spyware Logic Bombs Independent Trojan Horses Viruses Most current malicious code mixes some or all types Bacteria Worms Replicate Bowles and Pelaez

Historical Perspective ¨ Original computer virus idea – Fred Cohen, MIT 1984 – A few primitive virus-like programs existing beforehand ¨ First viruses – Spreading slowly, usually infecting floppy boot sector – Jerusalem – Later appended to shared programs ¨ Faster infection – Worms – E-mails, mobile code in browsers ¨ Intrusion and control – Taking control of infected machine – Trojans, spyware, zombies, rootkits ¨ “Commercialization” – Spy and espionage – Harvest information – Cyberterror(? ? )

Trap Doors (aka Back Doors) ¨ Secret part of the program that allows entry without going through the normal security procedures ¨ Debug code within legitimate application – Presumably to support maintenance – Reputable vendors wont take the risk of not telling you – Trap doors put by individual programmers can be caught in QA ¨ Trojans within appealing applications and utilities – Personal firewall can detect a request for service ¨ Special applications and data planted by an intruder – Usually will try to alter anti-virus and/or firewall policy to avoid detection

Adware and Spyware ¨ Adware – Started as advertising banners within free software – Can usually remove advertising if you pay software license ¨ Spyware – Usually, some free software will also collect information about you – Primarily surfing habits, cookies, etc. but who knows what else… – Also, sometimes you are essentially running a server on your machine which can serve for further penetration ¨ Spyware-like activity by legitimate software vendors – Designed to facilitate auto-update and version synchronization – Some record various characteristics of the client machine ¨ Corporate spies – Corporate spies may install software that records email, browsing, etc. • E. g. , Israeli Trojan at major corporations ¨ Solutions: – Use anti-spyware software (independent or part of OS) – Exit-control and information leakage software – Personal firewall can usually catch outgoing messages

Logic Bombs ¨ Logic bomb – A program that is set to explode when certain conditions occur – Examples: when the programmer is fired, on the expiration date of the license ¨ Easter Eggs – Hidden code that is inserted by the software programmers for fun or to show that they control the software (look at eeggs. com) – Usually not destructive (not really a bomb) ¨ Solutions: Independent QA testing, Code Review

Trojan Horses ¨ Malware hidden within another software – usually installed by a privileged user – when invoked may perform the unwanted function ¨ Malware impersonating another software – Replacing/hiding existing OS programs (rootkits) ¨ Malware sent by email that prompts the user to install it ¨ Malware installed when the user visits a web site – Russian mafia broke into legitimate web servers and planted trojans ¨ Examples: keystroke logger, DDo. S zombies, Net. Bus, rootkits ¨ Solutions: anti-virus, host-based IDSs, hardened OS, security policies, personal firewall

Example: Triggering Email

Net. Bus: One of the first Trojans

Viruses, Worms, and Bacteria ¨ Programs that replicate themselves over the network ¨ Viruses – Add own code to any uninfected copy of the host program – Replicates through exchange of programs between systems • MS-DOS viruses used. exe files as vehicle • Boot sector viruses also widespread ¨ Worms – An independent program that replicates itself over network connections, using email, or remote execution capabilities • • • Morris Worm crashed many Unix networks Klez is an email worm Code Red exploited IIS holes, mutated Kelvir spreads in IM networks SQL Slammer attacks MS-SQL servers

Speed of Infection (source: F-secure white paper)

Anti-Virus/Malware Approaches ¨ Detection modes – Scan incoming information (emails, communication ports, …) – Scan disk and memory for infected files ¨ Detection methodologies – Search for previously identified “signatures” • takes time for signatures to be discovered and distributed • viruses may compress themselves and the host program • polymorphic viruses change their signature – More sophisticated pattern recognition • identify parts of virus code and more general patterns • identify virus by its actions rather than its structure • emulate the execution of the virus until it decompresses itself – identify signature of uncompressed virus – identify virus behavior • 75 -90% of new (unknown) viruses can be discovered – Maintain cryptographic checksums of important files, to prevent alteration (Tripwire)

Anti-Virus Approaches ¨ Newer detection methods – Sandbox approach • execute the virus in separate partition • entrap the virus to infect and turn itself in – Digital Immune System (IBM – now Symantec) • Centralized identification of new viruses • Automated distribution of detection and fix to others on network – Market trends: move from Desktop to Server to Both ¨ Removal of the threat – remove the virus from the infected program – quarantine infected programs and content ¨ Prevention – Avoid disks, downloads, from unrecognized / uncertified sources – Use an anti-virus program to scan all new content ¨ Hoaxes – Some hoaxes disguise a virus as a solution

Study of Off-the-shelf Anti Virus Software (2006) ¨ Tested by AV-Test against 59000 backdoors, 70, 000 bots, and 160000 Trojans – Five vendors scored over 99% – Four tested below 50% – Median only 90. 42% ¨ One conclusion: use multi-layers – – Outgoing mail Server side ISP scanners And finally client… ¨ Many security professionals use multiple AVs

Intrusion Detection and Prevention Main Sources: Network Intrusion Detection / Northcutt, Novak The Honeynet Project

Intruders ¨ Who – – – Internal users (70% !!) Opponents (corporate, political) Vandals Kids (mentally) Criminals! ¨ Why – Stealing your money • Credit cards, passwords to bank accounts, extortion and blackmail – Other profit: • trade secrets, classified information, free usage of service – Vandalism • erase / modify information, interruption of service – Show off – Take control of a machine as a stepping stone to attacking another

Intrusion Methods ¨ Insecure system configuration – default configurations, password cracking, trust between systems, trust between application and database (access control) – Unpatched machines (about half of all PCs) – zero-day attack ¨ Software bugs – buffer overflows, sql injections, unexpected input combinations, race conditions ¨ Flaws in security protocols or their use – sniffing on wireless communication ¨ Social engineering – Take advantage of human weaknesses ¨ Trojan methods – getting a program running on a privileged user machine

Intrusion Scenarios Traditionally 1. Collect Intelligence to identify vulnerabilities • • 2. 3. Publicly available information Scan network for IP addresses, open ports, OS/services, patche • • • Perform intended activity Install backdoor future use Erase intrusion marks Run “exploit” scripts to penetrate Run additional exploits Today • Use phishing to get user to run scripts • Use trojans and loggers • Massively apply exploits against unpatched devices

Examples of Reconnaissance ¨ Ping sweeps – identify which machines are alive ¨ TCP and UDP scans – identify open ports ¨ ICMP queries – OS identification, including patch versions ¨ Account scans – break into weak user accounts (passwords!)

Example: LANGuard Scanning Tool

Examples of Exploits ¨ Hackers try to identify any type of service or program that can be exploited ¨ CGI Scripts – pass tainted input to shell, ask for a file ¨ Web server attacks (and SQL server, and other servers) – Sql injections, specially crafted bad inputs – buffer overflow ¨ Web client (browser) – execute mobile code – hiding 1 x 1 pixel frames within a trusted web page ¨ SMTP, and IMAP attacks – bugs in sendmail (SMTP) and in IMAP servers ¨ IP Spoofing – Do. S attacks, masquerading ¨ DNS Attacks – poisoning through masquerading and abuse of answer caching

Intruder Tools: ICMP Scanning ¨ ICMP can be used to identify live hosts and open ports ¨ Scan 1: send range of echo requests – Randomly – To a subnet broadcast IP: usually 255, 0 in BSD ¨ Scan 2: send range of address mask requests – routers will respond with info on the network addressing space ¨ Scan 3: use traceroute to collect “host unreachable” and “port unreachable” responses ¨ Solutions: – disable traffic to broadcast IPs (also helps in Do. S attacks) – disable ICMP echo – But, will not have ping, traceroute capabilities

Intruder Tools: DNS Attacks ¨ Use DNS queries to map the devices in attacked domain. – Responses may sometimes provide additional info (e. g. , OS) ¨ Use DNS zone transfer to get full dump ¨ Use traceroute to identify “live” hosts and possible routes ¨ DNS cache poisoning – A query from a compromised host that includes tainted input in the response field can result in DNS poisoning • this was used in the Clinton-Giuliani senate race to divert traffic from hillary 2000. com to hillaryno. com ¨ Solutions – BIND 8. 3 and up includes DNS Security Extensions (DNSSEC), requiring stronger authentication of querying servers – Limit zone transfers to few known servers – Do not fill DNS records with extra information

Intruder Tools: Masquerading ¨ Typical scenario – – Identify and characterize trust relationship between two systems Attack one system, e. g. using Do. S, or by crashing it Impersonate the silenced system to perform the crime Cover the crime scene ¨ Example: Mitnick Attack – – – Identified trusted TCP relationship between X-terminal and Server Silenced the Server using SYN flooding Do. S attack Hijacked TCP connection to x-terminal, by spoofing as Server IP Changed trusted domain to “all”, so that can access later Logged out, and stopped Server flooding, to avoid detection ¨ Solutions – Identify and “drop” SYN floods – Avoid trust relationships – Deploy tools that detect changes to critical files (e. g. tripwire)

Intruder Tools: Buffer Overflow ¨ A favorite of intruders, taking advantage of ill- implemented protocols and software – Most recent attacks use buffer overflow, e. g. , slammer ¨ Scenario: – transfer more data than the receiving party has reserved space for – embed your code, in the hope that it will be placed in the receiver program code space, and subsequently executed (use nops to avoid exact calculation) ¨ A double-whammy for “kiddy” intruders: intrusion success + demonstrated superiority over other programmers ¨ Solutions: – Vendor patches – More importantly: educate programmers and run code reviews – New CPUs allow marking data areas as non-executable

Intrusion Detection ¨ Goals: – – prevent damage prevent illegitimate use of resources, access to information, etc. catch and deter intruders identify weaknesses to improve defenses ¨ Important to detect before intruder gains access – most detection efforts concentrate on intelligence gathering scans – Common assumption: Intruders behave differently ¨ Unfortunately, easier to identify after the fact (forensic) ¨ Methods: – – rule-based detection, using known patterns (signatures) Log analsis; Statistical anomaly analysis most systems use combination of both System integrity verifiers (Tripwire), honey pots

NIDS and HIDS ¨ Network-based IDSs (NIDS) consist of agents that monitor network traffic ¨ Host-based IDSs (HIDS) consist of agents that monitor a single system ¨ Most deployments consist of – multiple instances of NIDS and HIDS – a centralized Console, as part of Security Operations Center (SOC) ¨ Use baselining to reduce false positives

Honeypots ¨ A honeypot is a decoy system that is placed within the internal network and is designed to attract intruders – – Usually configured as a characteristic system in same network May contain decoy and made-up data A little weaker than other systems IDS installed, and carefully monitored ¨ Goals: – Deception – Detection and advance warning (usually the actual attack is preceded by intelligence gathering) ¨ Another variation is an externally placed “scout” with bogus information – May identify itself to the outside as any of the network servers – Will continue the exchange with the hacker, and will alert

And Honeynets ¨ A honeynet is a decoy network of characteristic systems – Serves mostly for research and identification of new attacks – No production traffic is expected ¨ Example of a honeynet configuration – Internal nodes: characteristic systems, as-is without hardening – All traffic is recorded forensic analysis by firewall, IDS, and by the applications (data capture) – Firewall and router prevent actual usage of compromised systems for attack on external nodes (data control) ¨ Honeynet advantages: – Since no traffic is expected to arrive into the honeynet, anything that comes in is suspicious – Similarly, no traffic is expected to leave the net, hence attacks can be mitigated by blocking excessive traffic

Denial of Service Main Sources: CMU CERT, Riverhead, Northcutt et al

Denial Of Service (DOS) ¨ Interruption: prevent legitimate users access to a service ¨ Typical motives: political hacktivism, show off, blackmail ¨ Typical methods – Flooding of a network – Prevent connection between two or more machines – Prevent connection between one machine (server or client) and other machines – Crash a system, or network configuration ¨ Examples: – – Echo, SYN attacks on web servers, and ISPs Worms aimed at crashing email servers Attacks on specific application services DNS attacks that also divert traffic

DOS Example: SYN Attack (Network Layer) ¨ Works at TCP layer – Normal TCP 3 -way handshake: SYN, followed by SYN/ACK, followed by ACK – After SYN/ACK, server maintains an open connection until ACK is received ¨ Attack: flood server with IP spoofed SYNs until server exceeds the number of open connections allowed – Server will not be able to service legitimate users and may crash ¨ Detection: too many open SYN requests ¨ Prevention: – To protect others, do not allow outgoing packets to have a source that is not from your network (i. e. is spoofed) – Can also be done by the ISP

DOS Example: E-Mail Spamming (App Layer) ¨ Use email servers to clog themselves and other email servers – Code Red, Love Letter ¨ Attack: Send a large email to a large number of recipients – Directly to lists – Email worms ¨ Prevention: – Identify source and block its packets – Prevent large emails, large distribution lists

Other DOS Attacks ¨ Crashing a server by using a faulted implementation of a communication protocol – Teardrop uses improperly implemented TCP fragment reassembly – Land sends SYN packets with same source and destination addresses – Ping of Death sends oversized (>65 K) ICMP command ¨ Intentionally generating errors that are logged by the server to clog files and consume disk space ¨ Placing large files on ftp areas or network shared storage ¨ Generating excessive logins until system blocks logins from legitimate users – Many OS will block an account after 3 -5 failed login attempts

Distributed DOS (DDOS) ¨ Attack: Orchestrated from multiple sources at same time ¨ Solution: Identify packets addressed to attacked server, divert other traffic, and filter “dirty” packets

Examples of DDOS ¨ ICMP/UDP floods: TFN, Trinoo ¨ Code Red is a worm that has flood period from 20 th to 27 th of each month ¨ Social engineering is often used to get people to download DDOS daemons (zombies) ¨ In some cases, the attacker collaborate, e. g. , coordinated Arab attacks on Israeli sites

Example: DDo. S for Hire

DDo. S Illustration Zombies on innocent computers Infrastructure-level DDo. S attacks Bandwidth-level DDo. S attacks Server-level DDo. S attacks Source: Riverhead

Solution Overview Riverhead Guard Traffic destined to the victim 4. Filter only the bad Legitimate traffic to victim 5. Forward the legitimate 6. Non Victim traffic free flows Non-victimized servers Victim “No Dynamic configuration” Source: Riverhead

Adaptive and Dynamic Filtering 1 to 1000 s of dynamic filters by flow, protocol, … Static & Dynamic Filters Filter: Block spoof packets: traffic. Anomaly recognition, Drop non-essential E. g. , ICMP, UDPDNS, a. UDP line. etc. Per flow, using base TCP, if not essential, Anti spoofing Statistical analysis Layer 7 http smtp Per flow queues and aggregate rates Rate-limiting & DDo. S Traffic Shaping Source: Riverhead

ISP Perimeter Protection Source: Riverhead

ISP Perimeter Protection Source: Riverhead