Скачать презентацию Security Monitoring Thru Log Analysis Sam NG PISA Скачать презентацию Security Monitoring Thru Log Analysis Sam NG PISA

4647dfdaecc810916ef327e614db50a1.ppt

  • Количество слайдов: 42

Security Monitoring Thru Log Analysis Sam. NG@PISA Security Monitoring Thru Log Analysis Sam. [email protected]

What the hacker did Startup script was modified, a line is added to the What the hacker did Startup script was modified, a line is added to the rc. local n. /usr/bin/t 0 x 5 mm (I can’t remember the exact name) n “ls” the file, nothing showed n I tried to remove the file by “rm”, “rm” ran without any error n Many months later, I knew that’s something call “rootkit” n

Log was gone! n n Some of the log files are truncated I wanted Log was gone! n n Some of the log files are truncated I wanted to know how the hacker got in, and I redirected syslog to “/dev/lp 0” and ran for a couple of weeks Several inches of paper printed, but there is more than I could handle I can’t remember how, but finally I knew the hacker got in by exploiting an IMAP 4 vulnerability

Lesson Learnt n n You will need the log in sometime And better still, Lesson Learnt n n You will need the log in sometime And better still, store it in a safe place Write once read many (WORM) is good, but searching capability is even more important And the most important: you have to prepare it beforehand

But how can I use the log for security monitoring? But how can I use the log for security monitoring?

The Challenge “I don’t have the staff to handle 140, 000 alerts. I don’t The Challenge “I don’t have the staff to handle 140, 000 alerts. I don’t even have the staff to handle 12, 000 alerts” David Mac. Leod, Ph. D. , CISSP The Regence Group CISO

Reference: Counterpane’s Security Monitoring Service Bruce Schneier Reference: Counterpane’s Security Monitoring Service Bruce Schneier

Block Diagram Phase I Log Collection Phase II Noise Reduction Phase III Data Mining Block Diagram Phase I Log Collection Phase II Noise Reduction Phase III Data Mining Phase IV Alert/Ticket Management

Alert/Ticket Management Noise Reduction Log Collectors Data Mining Alert/Ticket Management Noise Reduction Log Collectors Data Mining

Log Collectors Send your log to DATABASE n Can be direct (open database connection Log Collectors Send your log to DATABASE n Can be direct (open database connection from the client) or in-direct (e. g. post data to a web application) n Each different log may need a different log collector n Many logs have built-in support to store data in database n

Unix syslog n n n n syslog can be redirected Can be redirected to Unix syslog n n n n syslog can be redirected Can be redirected to printer! (“/dev/lp 0” in Linux) Can be redirected to remote syslog host But redirect to program (pipe) seems not working!!? ? syslog-ng support logging directly to database Win 32 solutions to accept remote syslog records and log to database Plain text format, easy to write a program to watch the data and then send to database

Demo n A smiple “tail” base perl script to monitor new records in syslog Demo n A smiple “tail” base perl script to monitor new records in syslog and send it to remote database

Windows Eventlog Microsoft logparser can log to database n WMI interface allows you to Windows Eventlog Microsoft logparser can log to database n WMI interface allows you to query remote eventlog n Many third party solutions support logging to database n

Demo n A smiple C# program to monitor new records in Eventlog and send Demo n A smiple C# program to monitor new records in Eventlog and send it to remote database

Quiz 1 Which of the following Windows Server will pass Microsoft Baseline Analyzer (MSBA) Quiz 1 Which of the following Windows Server will pass Microsoft Baseline Analyzer (MSBA) 2. 0 auditing requirements with default install? A. B. C. D. Windows NT Windows 2000 Windows 2003 None of the above

DO YOU KNOW? n n n You can pass MSBA 2. 0 if you DO YOU KNOW? n n n You can pass MSBA 2. 0 if you enabled Success and Failure auditing for the Account Logon Events But indeed, it recommends you to enable Success and Failure auditing: Account Logon Events, Account Management, Policy Change, System Events Failure auditing: Directory Service Access, Object Access Data volume may be quite high, a server config in this way may generate a eventlog for every 2 seconds (actual volume depends on your server)

Microsoft IIS Web Server Native support log to ODBC (but seems to be only Microsoft IIS Web Server Native support log to ODBC (but seems to be only available on server platform only)

Microsoft IIS Web Server (2) n n n Microsoft does not recommend logging to Microsoft IIS Web Server (2) n n n Microsoft does not recommend logging to database if the IIS is busy (Q 245243) But nowadays most web pages are generated with SQL queries Test ODBC logging on your own environment Prep. Web. Log (Q 296093) convert l. IS log to SQL insertion statements in plain text format, but still have to run these SQL insertion by some other means Still, the best would be realtime logging to database Writing a “tail -f” similar program in IIS is difficult

DO YOU KOWN? IIS log file is updated in 64 k chunks. On servers DO YOU KOWN? IIS log file is updated in 64 k chunks. On servers that do not have a high usage rate, the statistics will not be up-to-date because of the delayed write (Q 142557) n When it flush, may be only first 20 k of the chunk contains data, the rest 44 k data is ‘’ and will be filled in next flush n Obviously a mmapped file, may be due to performance consideration… but n

Suggestions to programmer In C/C++, STDERR, cerr are not buffered n In fact, error Suggestions to programmer In C/C++, STDERR, cerr are not buffered n In fact, error log should be send out immediately, should not be cached/buffered n # man stderr …… …… CONSIDERATIONS The stream stderr is unbuffered. The stream stdout is linebuffered when it points to a terminal. Partial lines will not appear until fflush(3) or exit(3) is called, or a newline is printed. This can produce unexpected results, especially with debugging output. …… ……

J 2 EE Application Server Most application server support log 4 j or java. J 2 EE Application Server Most application server support log 4 j or java. util. logging (JDK 1. 4+) n log 4 j natively support logging to database thru JDBCAppender n

My Experience n n n n I have a program developed since JDK 1. My Experience n n n n I have a program developed since JDK 1. 3 At that time, Java don’t have java. util. logging And I don’t know much about log 4 j I wrote my own log handling routine (similar to syslog) to insert my own records into database But is not configurable/adaptable, and is not consolidated with the Application Server’s log Since logging is widely spread all over the codes, it is very difficult for me to change the program to use these new technology Log architecture should be planned in DESIGN PHASE

Snort IDS Comes with database output plugin to send alerts to database n The Snort IDS Comes with database output plugin to send alerts to database n The packet analysis thread is also responsible for database insertion n

My Experience I have experience in using Snort to monitor a ~20 M Internet My Experience I have experience in using Snort to monitor a ~20 M Internet link, with database output plugin, default rules n CPU is just about 30%, seems good n When I changed to log to local file, CPU sharp jump to 100% n Barnyard seems to be a solution but it does not support database!! n

And many others Checkpoint FW 1: thru LEA enabled product http: //www. opsec. com/ And many others Checkpoint FW 1: thru LEA enabled product http: //www. opsec. com/ n Microsoft ISA: default support ODBC logging (KB 838710) n Microsoft Exchange: same as Eventlog n Apache httpd: mod_log_sql n

Noise Reduction n Very important n Noise will kill the system n n 1. Noise Reduction n Very important n Noise will kill the system n n 1. 2. 3. 4. And to improve overall performance Can be done on several layers ignore certain message at the log collection sender ignore certain message at the log collection receiver end delete (mark as ignore) at the database fine tune your IDS rule, firewall logging policy etc. .

Data Mining “Data processing using sophisticated data search capabilities and statistical algorithms to discover Data Mining “Data processing using sophisticated data search capabilities and statistical algorithms to discover patterns and correlations in large preexisting databases; a way to discover new meaning in data” From http: //www. tfd. com

Data Mining (2) n n The heart of Security Monitoring A board term, a Data Mining (2) n n The heart of Security Monitoring A board term, a general concept Utilize database queries to get the information you want Can be an external program, can be a SQL server schedule job

Data Mining Techniques Data Mining Attack Detection Attack Definition Event Correlation Statistical Analysis Anomaly Data Mining Techniques Data Mining Attack Detection Attack Definition Event Correlation Statistical Analysis Anomaly Detection Normal Definition Event Correlation Statistical Analysis

Attack Definition IIS Unicode Directory Traversal (cmd. exe) attack strings GET GET GET GET Attack Definition IIS Unicode Directory Traversal (cmd. exe) attack strings GET GET GET GET GET /. . %c 0%af. . /winnt/system 32/cmd. exe? /c+dir+c: /MSADC/root. exe? /c+dir /_mem_bin/. . %255 c. . /winnt/system 32/cmd. exe? /c+dir /_vti_bin/. %252 e/winnt/system 32/cmd. exe? /c+dir+c: /_vti_bin/. . %255 c. . /winnt/system 32/cmd. exe? /c+dir /_vti_bin/. . %c 0%af. . /winnt/system 32/cmd. exe? /c+dir+c: /_vti_bin/. . %c 0%af. . /winnt/system 32/cmd. exe? /c+dir+c: /_vti_cnf/. . %c 0%af. . /winnt/system 32/cmd. exe? /c+dir+c: /adsamples/. . %c 0%af. . /winnt/system 32/cmd. exe? /c+dir+c: /cgi-bin/. . %255 c. . /winnt/system 32/cmd. exe? /c+dir+c: / /cgi-bin/. . %c 0%af. . /winnt/system 32/cmd. exe? /c+dir+c: /cgi-bin/. . %f 0%80%80%af. . /winnt/system 32/cmd. exe? /c+dir+c: / /iisadmpwd/. . %c 0%af. . /winnt/system 32/cmd. exe? /c+dir+c: /iisadmpwd/. . %c 0%af. . /winnt/system 32/cmd. exe? /c+dir+c: /msadc/. %252 e/winnt/system 32/cmd. exe? /c+dir+c: /msadc/. . %c 0%af. . /winnt/system 32/cmd. exe? /c+dir+c: /msdac/root. exe? /c+dir+c: /scripts/. . %e 0%80%af. . /winnt/system 32/cmd. exe? /c+dir+c: /scripts/. . %f 0%80%80%af. . /winnt/system 32/cmd. exe? /c+dir+c:

Vulnerability characteristic ≠attack characteristic n n n Code. Red I [/default. idq? NNNNN……] Code. Vulnerability characteristic ≠attack characteristic n n n Code. Red I [/default. idq? NNNNN……] Code. Red II [/default. idq? XXXXX……] In fact, the vulnerability can be exploited if the variable name is around 240 bytes Regular Expression [/default. idq? . {240, }] Difficult to write an effective and accurate definition And not all attack leave audit trail

Event Correlation n n E. g. 1000 login failures followed by ONE successful login Event Correlation n n E. g. 1000 login failures followed by ONE successful login from the same IP E. g. IF http_response_code = 500; THEN find_all_other_url_accessed(); ENDIF

Anomaly Detection Anomaly Detection

Normal Definition You define what is normal and then monitor it n E. g. Normal Definition You define what is normal and then monitor it n E. g. Operators login time should be corresponding to their shift duty n E. g. All server services should not be restarted unless necessary (ignore service start within 3 minutes of system startup) n

Statistical Analysis n n E. g. on average a event occurs 10 time a Statistical Analysis n n E. g. on average a event occurs 10 time a day, with a standard deviation of 2. 3. But today we have 2000 records. E. g. anything happens more than 200 times in the past 30 minutes E. g. a event never seen in the past 7 days E. g. “TOP 10” events/users/hosts, etc…

Alert/Ticket Management n n n Works like Bug Tracker Save the alert as a Alert/Ticket Management n n n Works like Bug Tracker Save the alert as a “Ticket” A ticket is a something like an outstanding job Assign the ticket to a staff to follow up Escalate it if remain unresolved for some time And don’t forget People Management (Time and Skill)

Alert/Ticket Management (2) n n n According to my experience, at least 90% of Alert/Ticket Management (2) n n n According to my experience, at least 90% of the alerts generated by data mining are still FALSE ALARMS People will get use to it and tend to think EVERY alerts are false alarms If possible, fine tune the system to eliminate the false alarm from occurring again

Couterpane 2003 Results Couterpane 2003 Results

DO YOU KNOW? n TSL providing Email to pager service for about HK$80/mth DO YOU KNOW? n TSL providing Email to pager service for about HK$80/mth

How should I start? n n n Do it step by step, phase by How should I start? n n n Do it step by step, phase by phase, Event. Log, syslog are easy to start with A group brain storming section would give you at least 10 such data mining rules, and is a good starting point Security Monitoring is a (long term) process, do not regard it as a single one-shot install and forget project

Contact: samngms@yahoo. com Contact: [email protected] com