C Network Programming Mastering Complexity with ACE

C++ Network Programming Mastering Complexity with ACE & Patterns Dr. Douglas C. Schmidt d. schmidt@vanderbilt. edu www. cs. wustl. edu/~schmidt/tutorials-ace. html Professor of EECS Vanderbilt University Nashville, Tennessee

Introduction (1/3) u ACE § Adaptive Communication Environment u The key contribution: Creation of uniform models which capture a broad spectrum of services § Easier to do on a specific domain § Easier to do when dealing with small building blocks • For instance: The scoped-locking constructs u The motivation: Development of a concurrent server program (socket-based) 2

Comments (2/3) u Huge scope § General overview § Specific inspection of certain parts § Our goal: A brief introduction § (Slides per minute – High rate) u API Specifications (Doxygen) § http: //www. dre. vanderbilt. edu/Doxygen/Current/html/a ce/hierarchy. html 3

Introduction (3/3) u ACE is somewhat outdated § does not use exceptions u C++ oriented § Some of the ideas seem trivial to Java programmers § Some of the ideas cannot be supported on Java § Thru ACE we can understand the design rationale § Useful for C++ developers § Useful for any developer of concurrent/distributed programs u Example: Java’s thread class is a realization of ACE’s Active Objects pattern § (see next slide) 4

Active Object: intent Decouples method execution from method invocation and simplifies synchronized access to shared resources by concurrent threads 5

Terminology u Framework u Toolkit u Patterns 6

Motivation: Challenges of Networked Applications Observation • Building robust, efficient, & extensible concurrent & networked applications is hard • e. g. , we must address many complex topics that are less problematic for nonconcurrent, stand-alone applications Complexities in networked applications Accidental Complexities • Low-level APIs • Poor debugging tools • Algorithmic decomposition • Continuous re-invention/discovery of core concepts & components Inherent Complexities • Latency • Reliability • Load balancing • Causal ordering • Scheduling & synchronization • Deadlock 7

Key Capabilities Provided by ACE Service Access & Control Concurrency Event Handling & IPC Synchronization 8

The Layered Architecture of ACE www. cs. wustl. edu/~schmidt/ACE. html Features • Open-source • 200, 000+ lines of C++ • 40+ personyears of effort • Ported to many OS platforms 9

The Pattern Language for ACE Pattern Benefits • Preserve crucial design information used by applications & middleware frameworks & components • Facilitate reuse of proven software designs & architectures • Guide design choices for application developers 10

The Frameworks in ACE Framework Inversion of Control Reactor & Proactor Calls back to application-supplied event handlers to perform processing when events occur synchronously & asynchronously Service Configurator Calls back to application-supplied service objects to initialize, suspend, resume, & finalize them Task Calls back to an application-supplied hook method to perform processing in one or more threads of control Acceptor-Connector Calls back to service handlers to initialize them after they are connected Streams Calls back to initialize & finalize tasks when they are pushed & popped from a stream 11

Example: Applying ACE in Real-time Avionics Goals • Apply COTS & open systems to missioncritical real-time avionics Key System Characteristics • Deterministic & statistical deadlines • ~20 Hz • Low latency & jitter • ~250 usecs • Periodic & aperiodic processing • Complex dependencies • Continuous platform upgrades Key Results • Test flown at China Lake NAWS by Boeing OSAT II ‘ 98, funded by OS-JTF • www. cs. wustl. edu/~schmidt/TAO-boeing. html • Also used on SOFIA project by Raytheon • sofia. arc. nasa. gov • First use of RT CORBA in mission computing • Drove Real-time CORBA standardization 12

Limitations with the Socket APIs (1/2) Poorly structured, non-uniform, & non-portable • API is linear rather than hierarchical • i. e. , the API is not structured according to the different phases of connection lifecycle management and the roles played by the participants • No consistency among the names • Non-portable & error-prone • Function names: read() & write() used for any I/O handle on Unix but Windows needs Read. File() & Write. File() • Function semantics: different behavior of same function on different OS e. g. , accept () can take NULL client address parameter on Unix/Windows, but will crash on some operating systems, such as Vx. Works • Socket handle representations: different platforms represent sockets differently e. g. , Unix uses unsigned integers whereas Windows uses pointers • Header files: Different platforms use different names for header files for the socket API 13

Limitations with the Socket APIs (2/2) Lack of type safety • I/O handles are not amenable to strong type checking at compile time • e. g. , no type distinction between a socket used for passive listening & a socket used for data transfer Steep learning curve due to complex semantics • Multiple protocol families & address families • Options for infrequently used features such as broadcasting, async I/O, non blocking I/O, urgent data delivery • Communication optimizations such as scatter-read & gather-write • Different communication and connection roles, such as active & passive connection establishment, & data transfer Too many low-level details • Forgetting to use the network byte order before data transfer • Possibility of missing a function, such as listen() • Possibility of mismatch between protocol & address families • Forgetting to initialize underlying C structures e. g. , sockaddr • Using a wrong socket for a given role 14

Example of Socket API Limitations (1/3) 1 #include 2 #include Possible differences in header file names 3 4 const int PORT_NUM = 10000; 5 6 int echo_server () 7 { 8 struct sockaddr_in addr; 9 int addr_len; Forgot to initialize to sizeof (sockaddr_in) 10 char buf[BUFSIZ]; 11 int n_handle; 12 // Create the local endpoint. Use of non-portable handle type 15

Example of Socket API Limitations (2/3) 13 int s_handle = socket (PF_UNIX, SOCK_DGRAM, 0); 14 if (s_handle == -1) return -1; 15 16 Use of non-portable return value 17 // Set up address information where server listens. Protocol and address family addr. sin_family = AF_INET; mismatch 18 addr. sin_port = PORT_NUM; 19 20 addr. sin_addr = INADDR_ANY; Unused structure members not zeroed out 21 if (bind (s_handle, (struct sockaddr *) &addr, 22 23 24 Wrong byte order sizeof addr) == -1) return -1; Missed call to listen() 16

Example of Socket API Limitations (3/3) 25 // Create a new communication endpoint. 26 if (n_handle = accept (s_handle, (struct sockaddr *) &addr, 27 &addr_len) != -1) { SOCK_DGRAM handle illegal here 28 int n; 29 while ((n = read (s_handle, buf, sizeof buf)) > 0) 30 write (n_handle, buf, n); 31 Reading from wrong handle No guarantee that “n” bytes will be written 32 close (n_handle); 33 } 34 return 0; 35 } 17

The Wrapper Facade Pattern (1/2) Context • Networked applications must manage a variety of OS services, including processes, threads, socket connections, virtual memory, & files Applications • OS platforms provide low-level APIs written in C to access these services Problem • The diversity of hardware & operating systems makes it hard to build portable & robust networked application software • Programming directly to low-level OS APIs is tedious, error-prone, & nonportable Solaris Win 2 K Vx. Works Linux Lynx. OS 18

The Wrapper Facade Pattern (2/2) Solution • Apply the Wrapper Facade design pattern to avoid accessing low-level operating system APIs directly Wrapper Facade Application This pattern encapsulates data & functions provided by existing non-OO APIs within more concise, robust, portable, maintainable, & cohesive OO class interfaces data calls method 1() … method. N() calls methods calls void method 1(){ function. A(); function. B(); } : Application API Function. A() API Function. B() API Function. C() void method. N(){ function. A(); } : Wrapper Facade : APIFunction. A : APIFunction. B method() function. A() function. B() 19

ACE Socket Wrapper Facade Classes ACE defines a set of C++ classes that address the limitations with the Socket API • Enhance type-safety • Ensure portability • Simplify common use cases • Building blocks for higher-level abstractions These classes are designed in accordance with the Wrapper Facade design pattern 20

Roles in the ACE Socket Wrapper Facade • The active connection role (ACE_SOCK_Connector) is played by a peer application that initiates a connection to a remote peer • The passive connection role (ACE_SOCK_Acceptor) is played by a peer application that accepts a connection from a remote peer & • The communication role (ACE_SOCK_Stream) is played by both peer applications to exchange data after they are connected 21

Connector/Acceptor: intent Decouples active/passive connection establishment from the service performed once the connection is established 22

The ACE_SOCK_Connector Class Motivation • There is a confusing asymmetry in the Socket API between (1) connection roles & (2) socket modes • e. g. , an application may accidentally call recv() or send() on a data-mode socket handle before it's connected • This problem can't be detected until run time since C socket handles are weakly-typed int buggy_echo_client (u_short port_num, const char *s) { int handle = socket (PF_UNIX, SOCK_DGRAM, 0); write (handle, s, strlen (s) + 1); sockaddr_in s_addr; Operations called in memset (&s_addr, 0, sizeof s_addr); wrong order s_addr. sin_family = AF_INET; s_addr. sin_port = htons (port_num); connect (handle, (sockaddr *) &s_addr, sizeof s_addr); } 23

The ACE_SOCK_Connector Class Capabilities • ACE_SOCK_Connector is factory that establishes a new endpoint of communication actively & provides capabilities to • Initiate a connection with a peer acceptor & then to initialize an ACE_SOCK_Stream object after the connection is established • Initiate connections in either a blocking, nonblocking, or timed manner 24

Using the ACE_SOCK_Connector • This example shows how the ACE_SOCK_Connector can be used to connect a client application to a Web server int main (int argc, char *argv[]) { const char *server_hostname = argv[1]; ACE_SOCK_Connector connector; ACE_SOCK_Stream peer; ACE_INET_Addr peer_addr; • Instantiate the connector, data transfer, & address objects if (peer_addr. set (80, server_hostname) == -1) return 1; else if (connector. connect (peer, peer_addr) == -1) return 1; • Block until connection established or connection request failure 25

The ACE_SOCK_Acceptor Class (1/2) Motivation • The C functions in the Socket API are weakly typed, which makes it easy to apply them incorrectly in ways that can’t be detected until run-time • The ACE_SOCK_Acceptor class ensures type errors are detected at compile-time int buggy_echo_server (u_short port_num) { sockaddr_in s_addr; int acceptor = socket (PF_UNIX, SOCK_DGRAM, 0); s_addr. sin_family = AF_INET; s_addr. sin_port = port_num; s_addr. sin_addr. s_addr = INADDR_ANY; bind (acceptor, (sockaddr *) &s_addr, sizeof s_addr); int handle = accept (acceptor, 0, 0); for (; ; ) { char buf[BUFSIZ]; ssize_t n = read (acceptor, buf, sizeof buf); if (n <= 0) break; Reading from wrong handle write (handle, buf, n); } } 26

The ACE_SOCK_Acceptor Class (2/2) Class Capabilities • This class is a factory that establishes a new endpoint of communication passively & provides the following capabilities: • It accepts a connection from a peer connector & then initializes an ACE_SOCK_Stream object after the connection is established • Connections can be accepted in either a blocking, nonblocking, or timed manner • C++ traits are used to support generic programming techniques that enable the wholesale replacement of functionality via C++ parameterized types 27

Using the ACE_SOCK_Acceptor • This example shows how an ACE_SOCK_Acceptor & ACE_SOCK_Stream can be used to accept connections & send/receive data to/from a web client extern char *get_url_pathname (ACE_SOCK_Stream *); int main () • Instantiate the acceptor, data transfer, & address objects { ACE_INET_Addr server_addr; • Initialize a passive ACE_SOCK_Acceptor acceptor; mode endpoint to ACE_SOCK_Stream peer; listen for connections on port 80 if (server_addr. set (80) == -1) return 1; if (acceptor. open (server_addr) == -1) return 1; • Accept a new connection for (; ; ) { if (acceptor. accept (peer) == -1) return 1; peer. disable (ACE_NONBLOCK); // Ensure blocking . ACE_Auto_Array_Ptr pathname (get_url_pathname (peer)); ACE_Mem_Map mapped_file (pathname. get ()); • Send the requested } } data if (peer. send_n (mapped_file. addr (), mapped_file. size ()) == -1) return 1; peer. close (); • Close the connection to the sender return acceptor. close () == -1 ? 1 : 0; • Stop receiving any connections 28

Reactor: intent Decouples event demultiplexing and event handler dispatching from application services performed in response to events 29

Reactor * handle. Events() register. Handler() remove. Handler() IEvent. Handler handle. Event() get. Handle() Concrete. Handler Iterator handles = select(); while(handles. has. Next()) { Handle h = handles. next(); IEvent. Handler eh = table[h]; eh. handle. Event(); } handle. Event() get. Handle() 30

Proactor: intent Demultiplexes and dispatches service requests that are triggered by the completion of asynchronous events. 31

Proactor read() write() * <> IEvent. Handler handle. Event() Completion. Dispatcher completed() <> Concrete. Handler handle. Event() Async. IODevice read() write() 32

Half-Sync/Half-Async: intent Decouples synchronous I/O from asynchronous I/O in a system to simplify concurrent programming effort without degrading execution efficiency 33

The Half-Sync/Half-Async Pattern Sync Service Layer Queueing Layer Async Service Layer Sync Service 1 Sync Service 2 Sync Service 3 <> Queue <> <> <> Async Service External Event Source This pattern yields two primary benefits: 1. Threads can be mapped to separate CPUs to scale up server performance via multi-processing 2. Each thread blocks independently, which prevents a flowcontrolled connection from degrading the Qo. S that other clients receive 34

Half-Sync/Half-Async Pattern Dynamics : External Event Source : Async Service : Queue : Sync Service notification read() work() message enqueue() notification read() work() message • This pattern defines two service processing layers—one async & one sync—along with a queueing layer that allows services to exchange messages between the two layers • The pattern allows sync services (such as processing log records from different clients) to run concurrently, relative both to each other & to async/reactive services (such as event demultiplexing) 35

Drawbacks with Half-Sync/Half-Async Worker Thread 1 Worker Thread 2 Worker Thread 3 <> Request Queue <> <> handlers acceptor Event source §Problems: • Overhead when crossing inter-layer boundary • Lack of support for Async. operations in the sync. Layer • Solution: Leader/Followers pattern 36

The ACE_TSS Class (2/2) Class Capabilities • This class implements the Thread-Specific Storage pattern, which encapsulates & enhances the native OS Thread-Specific Storage (TSS) APIs to provide the following capabilities: • It supports data that are ``physically'' thread specific, that is, private to a thread, but allows them to be accessed as though they were ``logically'' global to a program • It uses the C++ delegation operator: operator->() • It encapsulates the management of the keys associated with TSS objects • For platforms that lack adequate TSS support natively (such as Vx. Works) ACE_TSS emulates TSS efficiently 37

The Thread-Specific Storage Pattern • The Thread-Specific Storage pattern allows multiple threads to use one ‘logically global’ access point to retrieve an object that is local to a thread, without incurring locking overhead on each object access manages thread 1 thread m key 1 [k, t] accesses Thread-Specific Object key n 38

Using ACE_TSS (1/3) • This example illustrates how to implement & apply ACE_TSS to our thread-per-connection logging server • In this implementation, each thread gets its own request count that resides in thread-specific storage to alleviate race conditions on the request count without requiring a mutex template TYPE * ACE_TSS: : operator-> () { if (once_ == 0) { // Ensure that we're serialized. ACE_GUARD_RETURN(ACE_Thread_Mutex, guard, keylock_, 0); if (once_ == 0) { ACE_OS: : thr_keycreate(&key_); once_ = 1; } } We used the double-checked locking optimization pattern here 39

Using ACE_TSS (2/3) TYPE *ts_obj = 0; // Initialize from thread-specific storage. ACE_OS: : thr_getspecific (key_, (void **) &ts_obj); // Check if this method's been called in this thread. if (ts_obj == 0) { // Allocate memory off the heap and store it in a pointer. ts_obj = new TYPE; // Store the dynamically allocated pointer in TSS. ACE_OS: : thr_setspecific (key_, ts_obj); } return ts_obj; } 40

Using ACE_TSS (3/3) class Request_Count { public: Request_Count (): count_ (0) {} void increment () { ++count_; } int value () const { return count_; } private: int count_; }; static ACE_TSS request_count; virtual int handle_data (ACE_SOCK_Stream *) { while (logging_handler_. log_record () != -1) // Keep track of number of requests. request_count->increment (); This call increments variable in thread-specific storage ACE_DEBUG ((LM_DEBUG, "request_count = %dn", request_count->value ())); } 41

Reminder: Using Window’s critical-section CRITICAL_SECTION cs; // Global variable void main() { if(!Initialize. Critical. Section(&cs)) return; // Create threads. . . Delete. Critical. Section(&cs) • Now, let’s think about a Critical. Section class… • How should its interface look like? } DWORD WINAPI Thread. Proc(LPVOID lp. Parameter) { Enter. Critical. Section(&cs); // Access the shared resource. Leave. Critical. Section(&cs); } 42

The ACE Synchronization Wrapper Facades • Different operating systems provide different synchronization mechanisms with different semantics using different APIs • Some of these APIs conform to international standards, such as Pthreads • Other APIs conform to de facto standards, such as Win 32 • Below we describe the following ACE classes that networked applications can use to synchronize threads and/or processes portably 43

The ACE_Lock* Pseudo-Class • The ACE mutex, readers/writer, semaphore, & file lock mechanisms all support the ACE_LOCK* interface shown below • ACE_LOCK* is a “pseudo-class, ” i. e. , it's not a real C++ class in ACE • We use it to illustrate the uniformity of the signatures supported by many of the ACE synchronization classes • e. g. , ACE_Thread_Mutex, ACE_Process_Mutex, & ACE_Thread_Semaphore 44

The ACE_Guard Classes Motivation • When acquiring and releasing locks explicitly, it can be hard to ensure that all paths through the code release the lock, especially when C++ exceptions are thrown • ACE provides the ACE_Guard class & its associated subclasses to help assure that locks are acquired & released properly Class Capabilities • These classes implement the Scoped Locking idiom, which leverages the semantics of C++ class constructors & destructors to ensure a lock is acquired & released automatically upon entry to and exit from a block of C++ code, respectively 45

The Scoped Locking Idiom Motivation • Code that shouldn’t execute concurrently must be protected by some type of lock that is acquired/released when control enters/leaves a critical section • If programmers must acquire & release locks explicitly, it is hard to ensure that the locks are released in all paths through the code • e. g. , in C++ control can leave a scope due to a return, break, continue, or goto statement, as well as from an unhandled exception being propagated out of the scope void method () { lock_. acquire (); // The implementation may return prematurely… lock_. release (); } • The Scoped Locking idiom defines a guard class whose constructor automatically acquires a lock when control enters a scope & whose destructor automatically releases the lock when control leaves the scope void method () { ACE_Guard guard (lock_); // The lock is released when the method returns } 46

Implementing Scoped Locking in ACE template Generic ACE_Guard Wrapper Facade class ACE_Guard { public: // Store a pointer to the lock and acquire the lock. ACE_Guard (LOCK &lock) : lock_ (&lock) { lock_->acquire (); } // Release the lock when the guard goes out of scope, ~ACE_Guard () { lock_->release (); } // Other methods omitted… private: // Pointer to the lock we’re managing. LOCK *lock_; }; • ACE_Write_Guard & ACE_Read_Guard acquire write locks & read locks, respectively • Instances of the ACE_Guard classes can be allocated on the run-time stack to acquire & release locks in method or block scopes that define critical sections 47

ACE Condition Variable Classes (1/2) Motivation • Condition variables allow threads to coordinate & schedule their processing efficiently • Condition variables are more appropriate than mutexes or semaphores when complex condition expressions or scheduling behaviors are needed • e. g. , condition variables are often used to implement synchronized message queues that provide “producer/consumer” communication to pass messages between threads Producer <> Thread Request Queue <> put() get() Consumer Thread uses 2 ACE_Thread_Condition ACE_Thread_Mutex wait() signal() broadcast() acquire() release() 48

ACE Condition Variable Classes (2/2) Class Capabilities • The ACE_Condition_Thread_Mutex uses the Wrapper Façade pattern to guide its encapsulation of process-scoped condition variable semantics • The ACE_Null_Condition is a zero-cost class whose interface conforms to the ACE_Condition_Thread_Mutex 49