OS Support for Building Distributed Applications Multithreaded Programming

OS Support for Building Distributed Applications: Multithreaded Programming using Java Threads Rajkumar Buyya Grid Computing and Distributed Systems (GRIDS) Laboratory Dept. of Computer Science and Software Engineering University of Melbourne, Australia http: //www. gridbus. org/~raj or http: //www. buyya. com 1

Outline n n n n Introduction Thread Applications Defining Threads Java Threads and States Architecture of Multithreaded servers Threads Synchronization Thread Concurrency Models Summary 2

Introduction n n We discuss how Middleware is supported by the Operating System (OS) facilities at the nodes of a distributed system. The OS facilitates: n n Encapsulation and protection of resources inside servers; It supports invocation of mechanisms required to access those resources including concurrent access/processing. 3

Middleware and Network Operating System (NOS) n Many DOS (Distributed OS) have been investigated, but there are none in general/wide use. But NOS are in wide use for various reasons both technical and nontechnical. n n n Users have much invested in their application software; they will not adopt to new OS that will not run their applications. The 2 nd reason against the adoption of DOS is that users tend to prefer to have a degree of autonomy of their machines, even in a closely knit organisation. A combination of middleware and NOSs provides an acceptance balance between the requirement of autonomy and network transparency. n n NOS allows users to run their favorite word processor. Middleware enables users to take advantage of services that become available in their distributed system. 4

Operating system layers and Middleware n Unix and Windows are two examples of Network Operating Systems – have a networking capability built into them and so can be used to access remote resources using basic services such as rlogin, telnet. 5

Core OS components and functionality 6

A single threaded program class ABC { …. public void main(. . ) { …. . } begin body end } 7

A Multithreaded Program Main Thread start Thread A start Thread B Thread C Threads may switch or exchange data/results 8

Single and Multithreaded Processes threads are light-weight processes within a process Single-threaded Process Multiplethreaded Process Threads of Execution Multiple instruction stream Single instruction stream Common Address Space 9

Multi-Processing (clusters & grids) and Multi-Threaded Computing Threaded Libraries, Multi-threaded I/O Application CPU CPU Better Response Times in Multiple Application Environments CPU CPU Higher Throughput for Parallelizeable Applications 10

Web/Internet Applications: Serving Many Users Simultaneously PC client Internet Server Local Area Network PDA 11

Multithreaded Server: For Serving Multiple Clients Concurrently Server Process Client 1 Process Server Threads n Internet Client 2 Process 12

Modern Applications need Threads (ex 1): Editing and Printing documents in background. Printing Thread Editing Thread 13

Multithreaded/Parallel File Copy reader() { - - - - lock(buff[i]); read(src, buff[i]); unlock(buff[i]); - - - - } buff[0] buff[1] writer() { - - - - - lock(buff[i]); write(src, buff[i]); unlock(buff[i]); - - - - - } Cooperative Parallel Synchronized Threads 14

Levels of Parallelism Sockets/ PVM/MPI Threads Compilers CPU Task i-l func 1 ( ) {. . . . } a ( 0 ) =. . b ( 0 ) =. . + Task i func 2 ( ) {. . . . } a ( 1 )=. . b ( 1 )=. . x Task i+1 func 3 ( ) {. . . . } a ( 2 )=. . b ( 2 )=. . Load Code-Granularity Code Item Large grain (task level) Program Medium grain (control level) Function (thread) Fine grain (data level) Loop (Compiler) Very fine grain (multiple issue) With hardware 15

Multithreading - Multiprocessors Process Parallelism CPU P 1 P 2 CPU P 3 CPU time No of execution process more the number of CPUs 16

Multithreading on Uni-processor n Concurrency Vs Parallelism K Process Concurrency P 1 P 2 CPU P 3 time Number of Simultaneous execution units > number of CPUs 17

What are Threads? n n A piece of code that run in concurrent with other threads. Each thread is a statically ordered sequence of instructions. Threads are being extensively used express concurrency on both single and multiprocessors machines. Programming a task having multiple threads of control – Multithreading or Multithreaded Programming. 18

Java Threads n n Java has built in thread support for Multithreading Synchronization Thread Scheduling Inter-Thread Communication: n n n current. Thread yield sleep resume start run stop set. Priority get. Priority suspend Java Garbage Collector is a low-priority thread. 19

Threading Mechanisms. . . n n Create a class that extends the Thread class Create a class that implements the Runnable interface 20

1 st method: Extending Thread class n Threads are implemented as objects that contains a method called run() class My. Thread extends Thread { public void run() { // thread body of execution } } n n n Create a thread: My. Thread thr 1 = new My. Thread(); Start Execution of threads: thr 1. start(); Create and Execute: new My. Thread(). start(); 21

An example class My. Thread extends Thread { // the thread public void run() { System. out. println(" this thread is running. . . "); } } // end class My. Thread class Thread. Ex 1 { // a program that utilizes the thread public static void main(String [] args ) { My. Thread t = new My. Thread(); My. Thread t 2 = new My. Thread(); // due to extending the Thread class (above) // I can call start(), and this will call // run(). start() is a method in class Thread. t 2. start(); t. start(); } // end main() } // end class Thread. Ex 1 22

Next Lecture n More about threads programming n n Using interfaces Multiple threads in a program 23

2 nd method: Threads by implementing Runnable interface class My. Thread extends ABC implements Runnable {. . . public void run() { // thread body of execution } } n Creating Object: My. Thread my. Object = new My. Thread(); n Creating Thread Object: Thread thr 1 = new Thread( my. Object ); n Start Execution: thr 1. start(); 24

An example class My. Thread implements Runnable { public void run() { System. out. println(" this thread is running. . . "); } } // end class My. Thread class Thread. Ex 2 { public static void main(String [] args ) { Thread t = new Thread(new My. Thread()); // due to implementing the Runnable interface // I can call start(), and this will call run(). t. start(); } // end main() } // end class Thread. Ex 2 25

Life Cycle of Thread new start() wait() sleep() suspend() blocked runnable stop() dead non-runnable notify() slept resume() unblocked 26

A Program with Three Java Threads n Write a program that creates 3 threads 27

Three threads example n n n n n n class A extends Thread { public void run() { for(int i=1; i<=5; i++) { System. out. println("t From Thread. A: i= "+i); } System. out. println("Exit from A"); } } class B extends Thread { public void run() { for(int j=1; j<=5; j++) { System. out. println("t From Thread. B: j= "+j); } System. out. println("Exit from B"); } } 28

n n n n class C extends Thread { public void run() { for(int k=1; k<=5; k++) { System. out. println("t From Thread. C: k= "+k); } n n n } } System. out. println("Exit from C"); class Thread. Test { public static void main(String args[]) { new A(). start(); new B(). start(); new C(). start(); } } 29

Run 1 [raj@mundroo] threads [1: 76] java Thread. Test From Thread. A: i= 1 From Thread. A: i= 2 From Thread. A: i= 3 From Thread. A: i= 4 From Thread. A: i= 5 Exit from A From Thread. C: k= 1 From Thread. C: k= 2 From Thread. C: k= 3 From Thread. C: k= 4 From Thread. C: k= 5 Exit from C From Thread. B: j= 1 From Thread. B: j= 2 From Thread. B: j= 3 From Thread. B: j= 4 From Thread. B: j= 5 Exit from B n 30

Run 2 [raj@mundroo] threads [1: 77] java Thread. Test From Thread. A: i= 1 From Thread. A: i= 2 From Thread. A: i= 3 From Thread. A: i= 4 From Thread. A: i= 5 From Thread. C: k= 1 From Thread. C: k= 2 From Thread. C: k= 3 From Thread. C: k= 4 From Thread. C: k= 5 Exit from C From Thread. B: j= 1 From Thread. B: j= 2 From Thread. B: j= 3 From Thread. B: j= 4 From Thread. B: j= 5 Exit from B Exit from A n 31

Partitioning: Process Parallelism n n int add (int a, int b, int & result) // function stuff int sub(int a, int b, int & result) // function stuff Processor IS 1 pthread t 1, t 2; pthread-create(&t 1, add, a, b, & r 1); pthread-create(&t 2, sub, c, d, & r 2); pthread-par (2, t 1, t 2); add Processor IS 2 sub Data a b r 1 c d r 2 MISD and MIMD Processing 32

Partitioning: Data Parallelism n n n Data sort( int *array, int count) //. . . pthread-t, thread 1, thread 2; “ “ pthread-create(& thread 1, sort, array, N/2); pthread-create(& thread 2, sort, array, N/2); pthread-par(2, thread 1, thread 2); Processor Sort IS Processor Sort SIMD Processing do “ “ dn/2 dn 2/+1 “ “ dn 33

Multithreaded Server Client Process Server Threads Client Process User Mode Kernel Mode Message Passing Facility 34

Architecture for Multithread Servers n n Multithreading enables servers to maximize their throughput, measured as the number of requests processed per second. Threads may need to treat requests with varying priorities: n n A corporate server could prioritize request processing according to class of customers. Architectures: n n Worker pool Thread-per-request Thread-per-connection Thread-per-object 35

Client and server with threads Thread 2 makes requests to server Thread 1 generates results Input-output Receipt & queuing T 1 Requests N threads Client Server n n In worker-pool architecture, the server creates a fixed pool of worker threads to process requests. The module “receipt and queuing” receives requests from sockets/ports and places them on a shared request queue for retrieval by the workers. 36

Alternative server threading architectures 37

Assignment 1: Multithreaded Dictionary Server – Demonstrate the use Sockets and Threads A Client Program What is the Meaning of (“Love”)? “Love” “A deep, tender, ineffable feeling of affection and solicitude toward a person ” Multithreaded Dictionary Server (try, dictionary. com) “Solicitude” A Client Program What is the Meaning of (“Love”)? “care or concern, as for the well-being of another” A Client Program in “C++” 38

Next Lecture n n Thread Synchronisation Thread Priorities 39

Accessing Shared Resources n Applications Access to Shared Resources need to be coordinated. n n n Printer (two person jobs cannot be printed at the same time) Simultaneous operations on your bank account. Can the following operations be done at the same time on the same account? n n n Deposit() Withdraw() Enquire() 40

Online Bank: Serving Many Customers and Operations PC client Internet Bank Server Local Area Network Bank Database PDA 41

Shared Resources n n n If one thread tries to read the data and other thread tries to update the same date, it leads to inconsistent state. This can be prevented by synchronising access to the data. Use “Synchronized” method: n n public synchronized void update() { n n … } 42

the driver: 3 rd Threads sharing the same object class Internet. Banking. System { public static void main(String [] args ) { Account account. Object = new Account (); Thread t 1 = new Thread(new My. Thread(account. Object)); Thread t 2 = new Thread(new Your. Thread(account. Object)); Thread t 3 = new Thread(new Her. Thread(account. Object)); t 1. start(); t 2. start(); t 3. start(); // DO some other operation } // end main() } 43

Shared account object between 3 threads class My. Thread implements Runnable { Account account; public My. Thread (Account s) { account = s; } public void run() { account. deposit(); } } // end class My. Thread class Your. Thread implements Runnable { Account account; public Your. Thread (Account s) { account = s; } public void run() { account. withdraw(); } } // end class Your. Thread class Her. Thread implements Runnable { Account account; public Her. Thread (Account s) { account = s; } public void run() {account. enquire(); } } // end class Her. Thread account (shared object) 44

Monitor (shared object access): serializes operation on shared object class Account { // the 'monitor' int balance; // if 'synchronized' is removed, the outcome is unpredictable public synchronized void deposit( ) { // METHOD BODY : balance += deposit_amount; } } public synchronized void withdraw( ) { // METHOD BODY: balance -= deposit_amount; } public synchronized void enquire( ) { // METHOD BODY: display balance. } 45

Thread Priority n In Java, each thread is assigned priority, which affects the order in which it is scheduled for running. The threads so far had same default priority (NORM_PRIORITY) and they are served using FCFS policy. n Java allows users to change priority: n Thread. Name. set. Priority(int. Number) n n n MIN_PRIORITY = 1 NORM_PRIORITY=5 MAX_PRIORITY=10 46

Thread Priority Example class A extends Thread { public void run() { System. out. println("Thread A started"); for(int i=1; i<=4; i++) { System. out. println("t From Thread. A: i= "+i); } System. out. println("Exit from A"); } } class B extends Thread { public void run() { System. out. println("Thread B started"); for(int j=1; j<=4; j++) { System. out. println("t From Thread. B: j= "+j); } System. out. println("Exit from B"); } } 47

Thread Priority Example class C extends Thread { public void run() { System. out. println("Thread C started"); for(int k=1; k<=4; k++) { System. out. println("t From Thread. C: k= "+k); } System. out. println("Exit from C"); } } class Thread. Priority { public static void main(String args[]) { A thread. A=new A(); B thread. B=new B(); C thread. C=new C(); thread. C. set. Priority(Thread. MAX_PRIORITY); thread. B. set. Priority(thread. A. get. Priority()+1); thread. A. set. Priority(Thread. MIN_PRIORITY); System. out. println("Started Thread A"); thread. A. start(); System. out. println("Started Thread B"); thread. B. start(); System. out. println("Started Thread C"); thread. C. start(); System. out. println("End of main thread"); } } 48

Thread Programming models Thread concurrency/operation models n n n The master/worker model The peer model A thread pipeline 49

The master/worker model Program Resources Workers task. X Files Databases Master task. Y Input (Stream) main ( ) Disks task. Z Special Devices 50

Example n n n n n n main() /* the master */ { forever { get a request; switch( request ) case X: pthread_create(. . , task. X); case Y: pthread_create(. . , task. Y); . . } } task. X() /* worker */ { perform the task, sync if accessing shared resources } task. Y() /* worker */ { perform the task, sync if accessing shared resources }. . --Above runtime overhead of creating thread can be solved by thread pool * the master thread creates all worker thread at program initialization and each worker thread suspends itself immediately for a wakeup call from the master 51

The peer model Program Input Resources Workers task. X Files Databases task. Y Disks task. Z Special Devices 52

$Example n n n n n main() { pthread_create(. . , thread 1. .$

Example n n n n n main() { pthread_create(. . , thread 1. . . task. X); pthread_create(. . , thread 2. . . task. Y); . . signal all workers to start wait for all workers to finish do any cleanup } } task. X() /* worker */ { wait for start perform the task, sync if accessing shared resources } task. Y() /* worker */ { wait for start perform the task, sync if accessing shared resources } 53

A thread pipeline Program Filter Threads Stage 1 Stage 2 Stage 3 Input (Stream) Resources Files Databases Disks Special Devices 54

$Example main() { pthread_create(. . , stage 1); pthread_create(. . , stage 2); .$

Example main() { pthread_create(. . , stage 1); pthread_create(. . , stage 2); . . wait for all pipeline threads to finish do any cleanup } stage 1() { get next input for the program do stage 1 processing of the input pass result to next thread in pipeline } stage 2(){ get input from previous thread in pipeline do stage 2 processing of the input pass result to next thread in pipeline } stage. N() { get input from previous thread in pipeline do stage N processing of the input pass result to program output. } 55

Java thread constructor and management methods Thread(Thread. Group group, Runnable target, String name) Creates a new thread in the SUSPENDED state, which will belong to group and be identified as name; the thread will execute the run() method of target. set. Priority(int new. Priority), get. Priority() Set and return the thread’s priority. run() A thread executes the run() method of its target object, if it has one, and otherwise its own run() method (Thread implements Runnable). start() Change the state of the thread from SUSPENDED to RUNNABLE. sleep(int millisecs) Cause thread to enter the SUSPENDED state for the specified time. yield() Enter the READY state and invoke the scheduler. destroy() Destroy the thread. 56

Java thread synchronization calls thread. join(int millisecs) Blocks the calling thread for up to the specified time until thread has terminated. thread. interrupt() Interrupts thread: causes it to return from a blocking method call such as sleep(). object. wait(long millisecs, int nanosecs) Blocks the calling thread until a call made to notify() or notify. All() on object wakes the thread, or the thread is interrupted, or the specified time has elapsed. object. notify(), object. notify. All() Wakes, respectively, one or all of any threads that have called wait() on object. 57

Scheduler activations 58

Invocations between address spaces 59

RPC delay against parameter size 60

A lightweight remote procedure call 61

Times for serialized and concurrent invocations 62

Summary n Operating system provides various types of facilities to support middleware for distributed system: n n Multithreading enables servers to maximize their throughput, measured as the number of requests processed per second. Threads support treating of requests with varying priorities. Various types of architectures can be used in current processing: n n n encapsulation, protection, and concurrent access and management of node resources. Worker pool Thread-per-request Thread-per-connection Thread-per-object Threads need to be synchronized when accessing and manipulating shared resources. New OS designs provide flexibility in terms of separating mechanisms from policies. 63