Abstracting the Content of System Call Traces Waseem

Abstracting the Content of System Call Traces Waseem Fadel Abdelwahab Hamou-Lhadj Department of Electrical and Computer Engineering Concordia University Montréal, QC, Canada {w_fadel, abdelw}@ece. concordia. ca 1

Objective of the Trace Abstraction and Analysis Track (Reminder) Develop trace abstraction techniques and tools to facilitate the understanding and analysis of the content of large event-based system call traces 2

Benefits l l l Help users understand the behavioural aspects of a system Allow automated comparison of multiple traces based on the system behaviour and not a mere set of events Monitor the system performance to detect service degradation Ensure that subsequent revisions of a software system have not introduced programming errors Compare traces of redundant servers, performing the same work in order to detect any malfunction possibly caused by a security breach 3

Proposed Approach Trace Generated from Linux Kernel (LTTng events) Trace Abstraction Algorithm Based on Pattern Matching and Filtering of Noise High-Level Trace 4

Pattern Library l We built a pattern library that contains several patterns that represent key Linux kernel operations l l File, socket and process management operations The patterns are modeled as state machines l l States represent system modes (user_mode, sys_call mode, etc. ) Events consist of LTTng events 5

Patterns we have so far l We completed the pattern library to include the following operations: l l l File Management (Open, Read, Write, Seek, Access, File Manipulation with fcntl, Stat, Close) Socket Management for both TCP and UDP (Create, Connect, Bind, Listen, Accept, Send, Receive, Close) Process Management (Execution with exec and execve, Exit, Fork, Clone, Get Resource Limit, Get Time of The Day, New UName, Unlink, Read Link) 6

Filtering of Trace Noise l We define noise in an LTTng trace as any event associated with memory management, page faults, and interrupts l l Are dependent on a specific kernel version Can occur anywhere in the trace and in any order Are treated similarly to the way utilities have been treated in related work Associated events are treated as a set l i. e. order of occurrence of detailed events is ignored 7

Validation of Patterns l The patterns have been validated by Pierre-Marc Fournier and Mathieu Desnoyers from École Polytechnique de Montréal l Regular meetings with them have also helped in the process of understanding the markers, the system calls, and the main modes of execution l Both users agreed with the way we defined noise found in traces 8

The Linux Kernel Trace Abstraction Tool l l The tool takes as input a trace generated from LTTng tracer It applies our approach to that trace Outputs the trace in its abstracted format It has been developed in Java as an Eclipse plug-in 9

The Linux Kernel Trace Abstraction Tool (cont. ) l The tool was designed to accept patterns defined as external XML files <? xml version="1. 0" encoding="ISO-8859 -1"? > <pattern name="Sample Pattern" type="High. Level. Sample. Constrcut" noise="false"> <event name="syscall_entry" syscall_name="sys_sample" order="1" prev_state="IGNORE"> <current_state>SYSCALL_SAMPLE</current_state> </event> <event name="sample" order="2" prev_state="SYSCALL_SAMPLE"> <current_state>SYSCALL_SAMPLED</current_state> </event> <event name="syscall_exit" order="LAST" prev_state="SYSCALL_SAMPLED"> <current_state>USER_MODE_SAMPLED</current_state> </event> </pattern> 10

The Linux Kernel Trace Abstraction Tool (cont. ) 11

The Linux Kernel Trace Abstraction Tool (cont. ) l The tool consists of three parts: l l The top part, through which the developer can provide the tool with the required information The middle part, displays the trace resulting from the abstraction process The bottom part, displays the information related to the original and abstracted traces The following slides show the architecture and the class diagram 12

Architecture l l l Partitioned both horizontally and vertically Easy to extend (to add new system calls patterns or even patterns for different calls) Easy to maintain (to modify existing patterns or the design itself) 13

Horizontal Partitioning l Horizontal partitioning is performed by defining the main domains of the system 14

Vertical Partitioning 15

Vertical Partitioning (cont. ) l l Divide the system into different layers Define the interfaces between layers Presentation layer can be developed without affecting lower layers (Multiple GUIs can be provided for the same data) Components from different layers can be designed, implemented and maintained independently 16

Class Diagram 17

Class Diagram (cont. ) l l l Adding new patterns and high-level constructs can easily be done by subclassing the appropriate classes and interfaces Multiple implementations representing different trace formats can be applied using the same interfaces High level constructs are easy to further abstract 18

Case Studies l l l We applied our approach to large traces generated while running different processes One process was the java virtual machine which was running a distributed file server and a client Another process was the eclipse framework Third process was gedit Fourth process was GIMP image editor The final process was firefox 19

Case Studies (cont. ) Process Initial size Size after Compression Abstraction Ratio Eclipse 1226985 325902 73% GIMP 847575 229778 73% Firefox 646710 257282 60% Gedit 186167 90482 51% JVM 47271 3033 93% 20

Case Studies (cont. ) A snapshot of the traces resulting from the abstraction of the JVM process SEQ(1) Socket Create: family = 2, type = 2, protocol = 0, sock = 0 xd 563 d 340, ret = 8 SEQ(1) Socket Connect: fd = 8, uservaddr = 0 x 80569 c 8, addrlen = 28, ret = 0 SEQ(2) File Stat: ip = 0 xb 7 f 78430, syscall_id = 221 [sys_fcntl 64+0 x 0/0 xb 0] SEQ(1) Get Time of Day: ip = 0 xb 7 f 78430, syscall_id = 78 [sys_gettimeofday+0 x 0/0 x 80] SEQ(1) Unknown Event: Event name: pollfd, Params: fd = 8 SEQ(1) Unknown Event: Syscall name: sys_poll, Params: ip = 0 xb 7 f 78430, syscall_id = 168 [sys_poll+0 x 0/0 xc 0] SEQ(1) Socket Send: call = 9, a 0 = 8 SEQ(4) Unknown Event: Event name: pollfd, Params: fd = 8 SEQ(1) Process Schedule: prev_pid = 0, next_pid = 23566, prev_state = 0 … SEQ(1) Unknown Event: Syscall name: sys_futex, Params: ip = 0 xb 7 f 78430, syscall_id = 240 [sys_futex+0 x 0/0 x 130] SEQ(1) Unknown Event: Syscall name: sys_clock_gettime, Params: ip = 0 xb 7 f 78430, syscall_id = 265 [sys_clock_gettime+0 x 0/0 xa 0] SEQ(1) Get Time of Day: ip = 0 xb 7 f 78430, syscall_id = 78 [sys_gettimeofday+0 x 0/0 x 80] SEQ(1) File Access: ip = 0 xb 7 f 78430, syscall_id = 33 [sys_access+0 x 0/0 x 30] SEQ(1) File Write: fd = 19 SEQ(1) Socket Receive: call = 10, a 0 = 19 21 SEQ(1) Process Exit: pid = 23580, cpu_id = 1, state = 1, Send Signal: pid = 23610, cpu_id = 1, state = 1, Sched Try Wakeup: pid = 23972, cpu_id = 1, state = 1

Remaining Challenges l Continuous improvement of the pattern library l l l Defining additional patterns Dealing with new LTTng events Using higher level constructs to further abstract the resulting traces Improving the algorithm in terms of performance Embedding the tool with the LTTv 22

Conclusion l l l We introduced techniques to abstract execution traces resulting from the Linux kernel Our approach is based on building a pattern library that consists of patterns of the most common operations in Linux We also defined noise patterns that result from memory management operations and page faults We introduced an algorithm to abstract the system call traces by using the pattern library We applied our techniques to traces generated from several processes 23