Reliable Scalable and Interoperable Internet Telephony Ph D

Reliable, Scalable and Interoperable Internet Telephony Ph. D thesis presentation by Kundan Singh Advisor: Henning Schulzrinne Computer Science Department, Columbia University, New York June 21, 2006

My research background/timeline SIP SP v r lato ion luat eva nce fere Con SIP ing P us Vo. I P 2 P ng hari ad s r/lo ove Fail SIP tion bora colla dia time ce Mul erfa r int use EMA AT CIN ile N Mob nse spo e re voic re ive uctu ract astr Inte infr o. IP g se V ncin y) rpri r fere Ente con libra SIP (SIP l + mai ip+ oice Libs -RT ure t lien 23 c H. 3 way gate cs 2 2006 2005 2004 2003 2002 2001 rk@ grad Wo rola er o Und a Mot di In @1997 1999 2000 MS@ & ol cs. c bility @ Relia Ph. D rans 23 t -H. 3 SIP ia Vo. IP uct astr infr b olum. c ity il alab sc a mbi u

Outline of the presentation Introduction n Server redundancy n n n Load sharing and failover in SIP telephony Comparison of thread models for SIP server Peer-to-peer (P 2 P) n n n SIP servers using external P 2 P network Additionally, P 2 P maintenance using SIP Enterprise IP telephony n n n What is the problem? Why important? My contributions Multi-platform collaboration using SIP Scalable centralized conferencing architecture Interworking between SIP/SDP and H. 323 Conclusion 36 slides 3

Telephone reliability & scalability (PSTN: Public Switched Telephone Network) database (SCP) for freephone, calling card, … signaling network (SS 7) local telephone switch (class 5 switch) signaling router 10, 000 customers (STP) 20, 000 calls/hour regional telephone switch (class 4 switch) 100, 000 customers 150, 000 calls/hour “bearer” network database (SCP) 10 million customers 2 million lookups/hour signaling router (STP) 1 million customers 1. 5 million calls/hour telephone switch (SSP) Switches strive for 99. 999% availability. Lucent’s 5 E-XC supports 4 million BHCA. 4

Internet telephony (SIP: Session Initiation Protocol) alice@yahoo. com example. com INVITE REGISTER INVITE 129. 1. 2. 3 bob@example. com 192. 1. 2. 4 DB DNS Can SIP server provide carrier grade reliability and scalability using commodity hardware 5

What are the problems? n Can SIP server provide carrier-grade reliability and scalability using commodity hardware? n n How can we build a server-less self-organizing peer-to-peer Vo. IP network? n n What affects the server performance? Can this be done in standards compliant way? Can communication be extended to multiplatform collaboration using existing protocols? n n How well multi-party conferencing scales? How to interoperate between SIP and H. 323? 6

My contributions n Server redundancy n n Peer-to-peer (P 2 P) n n n Implemented failover using database replication Two-stage architecture for SIP load sharing Comparison of thread models for SIP servers using external P 2 P network Additionally, P 2 P maintenance using SIP Enterprise IP telephony n n n Multi-platform collaboration using SIP Scalable centralized conferencing architecture Interworking between SIP/SDP and H. 323 New architecture, New algorithm or approach, Implementation, Evaluation 7

Outline of the presentation Introduction n Server redundancy n n n Load sharing and failover in SIP telephony Comparison of thread models for SIP server Peer-to-peer (P 2 P) n n n SIP servers using external P 2 P network Additionally, P 2 P maintenance using SIP Enterprise IP telephony n n n What is the problem? Why important? My contributions Multi-platform collaboration using SIP Scalable centralized conferencing architecture Interworking between SIP/SDP and H. 323 Conclusion 8

Server redundancy The problem: failure or overload INVITE REGISTER 9

High availability Master/ slave Implementation and analysis n Using My. SQL replication n TR Call setup latency DNS TTL, time-to-repair, retry timeout n P 2 P 1 System reliability individual component reliability n Caller D 2 D 1 DNS Implementation Slave/ master User unavailability Most registers are refreshes Retry timeout, replication interval, register refresh interval P 1 Tc P 2 D 1 D 2 Callee Td A Tc Tr TR A Tc 10

Scalability Load sharing: redundant proxies and databases P 1 n REGISTER n D 1 n n D 2 P 3 Write to D 1 & D 2 INVITE n P 2 INVITE REGISTER Read from D 1 or D 2 Database write/ synchronization traffic becomes bottleneck 11

Scalability Load sharing: divide the user space P 1 a-h n D 1 n P 2 i-q D 2 Proxy and database on the same host Stateless proxy can become overloaded n Use many P 3 r-z D 3 12

Scalability Comparison of the two designs P 1 a-h D 1 P 2 i-q D 2 P 3 D 2 r-z Low Scalability High Reliability D 2 High Scalability Low Reliability 13

Scalability (and reliability) Two stage architecture I stage II stage a*@example. com a 1 s 1 Master a 2 Slave a. example. com _sip. _udp SRV 0 0 a 1. example. com SRV 1 0 a 2. example. com sip: bob@example. com s 2 sip: bob@b. example. com b*@example. com s 3 ex example. com _sip. _udp SRV 0 40 s 1. example. com SRV 0 40 s 2. example. com SRV 0 20 s 3. example. com SRV 1 0 ex. backup. com b 1 Master b 2 Slave b. example. com _sip. _udp SRV 0 0 b 1. example. com SRV 1 0 b 2. example. com Capacity = f(#stateless, #groups) 14

Load Sharing Performance result: calls/second n n Using 3+3 servers gives carrier-grade performance (10 million busy hour call attempts) Registration test: supports 10 million subscribers On commodity hardware: 3 GHz, Pentium 4, 1 GB memory Test with UDP, stateless, no DNS and no mempool 15

Server performance What happens inside a server? What thread/event models possible? Web server recv accept SIP server stateful Response recvfrom parse Request File read send Match transaction Modify response Stateless proxy Found Match transaction Update DB REGISTER other Redirect/reject Lookup DB Stateless proxy 1. 2. 3. 4. (Blocking) I/O Critical section (lock) Critical section (r/w lock) Pure event-based (one thread) Thread-per-msg or transaction Pool-thread per msg (sipd) Two stage thread pool sendto Build response Proxy Modify Request DNS 16

Server performance Results of my measurement n n Event-based performs 30% better than existing thread-pool architecture on single-CPU Two stage thread-pool architecture gives better performance on multi-CPU n n 60% more on 4 x. Pentium Both Pentium and Sparc took 2 MHz of CPU cycles per call/s on single-CPU 17

Problem with servers C C S C n Server-based n C n P P P n Cost: maintenance, configuration Central points of failures, catastrophic failures Controlled infrastructure (e. g. , DNS) Peer-to-peer n n n Robust: no central dependency Self organizing, no configuration Inherent scalability 18

We built: P 2 P-SIP n Unlike proprietary Skype architecture n n Robust and efficient lookup using DHT Interoperability n n Hybrid architecture n n n Lookup in SIP+P 2 P Inter-domain P 2 P-SIP Unlike file-sharing applications n n n DHT algorithm uses SIP communication Data storage, caching, delay, reliability Data model and service model Disadvantages n Lookup delay and security 19

How to combine SIP + P 2 P? n SIP-using-P 2 P n n Replace SIP location service by a P 2 P protocol P 2 P-over-SIP n Additionally, implement P 2 P using SIP messaging SIP-using-P 2 P SIP proxies P 2 P-over-SIP Maintenance P 2 P SIP Lookup P 2 P SIP P 2 P network FIND INSERT INVITE sip: alice@128. 59. 194 INVITE alice P 2 P-SIP overlay REGISTER Alice 128. 59. 19. 194 20

SIP-using-P 2 P Logical Operations n Contact management n n Key storage n n n put (subscribee id, signed encrypted subscriber id) Composition needs service model Offline message n n User certificates and private configurations Presence n n put (user id, signed contact) put (recipient, signed encrypted message) NAT and firewall traversal n STUN and TURN server discovery needs service model XML-based data format 21

SIP-using-P 2 P Implementation in SIPc with the help of Xiaotao Wu n Open. DHT n n Identity protection n Using Data model Certificate-based SIP id == email P 2 P for Calls, IM, presence, offline message and name lookup 22

P 2 P-over-SIP Architecture and implementation n n DHT (Chord) algorithm using SIP messages with query and update semantics of REGISTER Has SIP registrar, proxy, user agent Other: discovery, NAT traversal, failover Adaptor: allows existing SIP devices to become P 2 P 23

P 2 P-over-SIP Analysis: scalability n Computed message load as function of n n Refresh rate (keep-alive, finger table, user registration), call arrival rate, churn (join, leave, failure), scale (number of peer nodes and users) Number of nodes = f(individual node capacity) n n Measured performance: 800 register/s. Assuming a conservative 10 reqister/s capacity, and aggressive refresh and call rate of 1/min, it gives more than 16 million peers (super nodes) in the network. 24

P 2 P-over-SIP Analysis: availability and call setup latency n To increase user availability: n n Fast failure detection: increase keep-alive rate Reduce unavailability: frequent registration refresh Replicate: user and node registrations Call setup latency: n Same as DHT lookup latency: O(log(N)) n n Calls to known locations (“buddies”) is direct Chord: 10000 nodes => 6 hops n n At most a few seconds User availability and retransmission timers 25

SIP-using-P 2 P vs P 2 P-over-SIP n n Not SIP-specific, hence no implementation overhead for non-Vo. IP but P 2 P applications Low transport and transaction overhead No P 2 P security burden on SIP No dependency on single DHT implementation n n Reuse SIP naming, routing, security, NAT/firewall traversal Easily reuse existing SIP components without change n n n voicemail, conference Single DHT implementation Readily supports service model 26

Server-based vs peer-to-peer Reliability, failover latency DNS-based. Depends on client retry timeout, DB replication latency, registration refresh interval DHT self organization and periodic registration refresh. Depends on client timeout, registration refresh interval. Scalability, number of users Depends on number of servers in the two stages. Depends on refresh rate, join/leave rate, uptime Call setup latency One or two steps. O(log(N)) steps. Security TLS, digest authentication, S/MIME Additionally needs a reputation system, working around spy nodes Maintenance, configuration Administrator: DNS, database, middle-box Automatic: one time bootstrap node addresses PSTN interoperability Gateways, TRIP, ENUM Interact with server-based infrastructure or co-locate peer node with the gateway 27

Outline of the presentation Introduction n Server redundancy n n n Load sharing and failover in SIP telephony Comparison of thread models for SIP server Peer-to-peer (P 2 P) n n n SIP servers using external P 2 P network Additionally, P 2 P maintenance using SIP Enterprise IP telephony n n n What is the problem? Why important? My contributions Multi-platform collaboration using SIP Scalable centralized conferencing architecture Interworking between SIP/SDP and H. 323 Conclusion 28

Internet telephony infrastructure CINEMA: Columbia Inter. Net Extensible Multimedia Architecture Telephone switch Local/long distance 1 -212 -5551212 CINEMA servers rtspd: media server sipconf: RTSP Conference server PSTN Department PBX Internal Telephone Extn: 7040 713 x Quicktime RTSP clients sipum: Unified messaging sipd: Proxy, redirect, Registrar server SIP/PSTN Gateway SQL database cgi vxml Web server SIP VXML Built many components in a complete system for enterprise IP telephony and multimedia collaboration siph 323: SIP-H. 323 translator Web based configuration My work H. 323 Net. Meeting 29

My other work n Communication to collaboration n n Centralized conferencing n n n Comprehensive, multi-platform collaboration using SIP Unified messaging: The gaps among different media (audio, video, text), devices (PC, phone) and means of communications (Email, SIP, IM) disappear for messaging Novel SIP/RTSP based voicemail and answering machine SIP interface to Voice. XML browser Audio mixing, video forwarding, IM, shared web browsing, screen sharing, web-based configuration and control, floor control Performance evaluation; cascaded server architecture SIP-H. 323 translation 30

Conference server Performance evaluation of audio mixer n On commodity PC n n n About 480 participants in a single conference with one active speaker About 80 four-party conferences, with one speaker each Both Pentium and Sparc took 6 MHz per participant 31

Conference server Cascaded architecture n n SIP REFER message is used to create cascading N. (N-1) participants Higher delay n n N 2/4 participants Lower delay I measured the CPU usage for two cascaded servers: supports about 1000 participants 32

Outline of the presentation Introduction n Server redundancy n n n Load sharing and failover in SIP telephony Comparison of thread models for SIP server Peer-to-peer (P 2 P) n n n SIP servers using external P 2 P network Additionally, P 2 P maintenance using SIP Enterprise IP telephony n n n What is the problem? Why important? My contributions Multi-platform collaboration using SIP Scalable centralized conferencing architecture Interworking between SIP/SDP and H. 323 Conclusion 33

Revisiting the problems n Developed a two stage scalable and Can SIP server provide carrier-grade reliability reliable SIP server architecture: linear scaling. Use event-based. and scalability using commodity hardware? n n How can we build a server-less self-organizing Developed P 2 P-SIP architecture: SIP peer-to-peer Vo. IP network? and P 2 P-over-SIP -using-P 2 P n n What affects the server performance? Can this be done in standards compliant way? Can communication be extended to multiplatform collaboration using existing Multi-platform collaboration using protocols? n n existing protocols and tools, How well multi-party conferencing scales? unified messaging, centralized conferencing (cascaded), SIPHow to interoperate between SIP and H. 323? H. 323 interworking. 34

Conclusions n Impact: n n n n Commercialized by SIPquest (now First. Hand) and sold to many customers. CINEMA was deployed in our department for a brief period of time. Used in various other projects at IRT: NG 911, firewall controller, presence scalability, TCP/TLS measurements, … P 2 P-SIP is a “hot” topic in industry and IETF now – client desktop, hardware phone as well as server vendors are pursuing this. SIP-H. 323 requirements eventually became an RFC Plan to open source SIPc for large scale deployment experience of P 2 P-SIP Started working on a P 2 P-based self organizing servers for 3 GPP at Bell Labs “So what” (Implications): n n Replacing PSTN – better features, quality and performance at lower cost and maintenance; zero cost Vo. IP using P 2 P-SIP Distributed, multi-provider, component architecture for telephony and collaboration 35

My publications Conference, workshop, technical report, magazine/journal 1. K. Singh and H. Schulzrinne, “Using an external DHT as a SIP location service", Columbia University Technical Report CUCS-007 -06, NY, Feb’ 06. 2. K. Singh and H. Schulzrinne, “Peer-to-peer Internet telephony using SIP", NOSSDAV, Skamania, Washington, Jun 2005. . K. Singh and H. Schulzrinne, "Peer-to-peer Internet Telephony using SIP", New York Metro Area Networking Workshop, CUNY, Sep 2004. K. Singh and H. Schulzrinne, "Peer-to-peer Internet Telephony using SIP", Columbia University Technical Report CUCS-044 -04, NY, Oct 2004. 3. K. Singh and H. Schulzrinne, “Failover, load sharing and server architecture in SIP telephony”, Elsevier Computer Communication Journal. To appear. Aug 2006. “K. Singh and H. Schulzrinne, “Failover and load sharing in SIP telephony", SPECTS (Symposium on performance evaluation of computer and telecommunication systems). Philadelphia, PA, Jul 2005. K. Singh and H. Schulzrinne, "Failover and Load Sharing in SIP Telephony", Columbia University Technical Report CUCS-011 -04, NY, May 2004. H. Schulzrinne, K. Singh and X. Wu, "Programmable Conference Server", Columbia University Technical Report CUCS-040 -04, NY, Oct 2004. 5. K. Singh, Xiaotao Wu, J. Lennox and H. Schulzrinne, "Comprehensive Multi-platform Collaboration", MMCN 2004 - SPIE Conference on Multimedia Computing and Networking, Santa Clara, CA, Jan 2004. 4. K. Singh, Xiaotao Wu, J. Lennox and H. Schulzrinne, "Comprehensive Multi-platform Collaboration", Columbia University Technical Report CUCS-027 -03, NY, Nov 2003. 6. M. Buddhikot, A. Hari, K. Singh and S. Miller, "Mobile. NAT: A new Technique for Mobility across Heterogeneous Address Spaces", ACM MONET journal, March 2005. M. Buddhikot, A. Hari, K. Singh and S. Miller, "Mobile. NAT: A new Technique for Mobility across Heterogeneous Address Spaces", WMASH 2003 - ACM International Workshop on Wireless Mobile Applications and Services on WLAN Hotspots, San Diego, CA, Sep 2003. 7. K. Singh, A. Nambi and H. Schulzrinne, "Integrating Voice. XML with SIP services", ICC 2003 - Global Services and Infrastructure for Next Generation Networks, Anchorage, Alaska, May 2003. K. Singh, A. Nambi and H. Schulzrinne, "Integrating Voice. XML with SIP services", Second New York Metro Area Networking Workshop, Columbia University, NY, Sep 2002. 8. K. Singh, W. Jiang, J. Lennox, S. Narayanan and H. Schulzrinne, "CINEMA: Columbia Inter. Net Extensible Multimedia Architecture", Columbia University Technical Report CUCS-011 -02, NY, May 2002. W. Jiang, J. Lennox, H. Schulzrinne and K. Singh, "Towards Junking the PBX: Deploying IP Telephony", NOSSDAV 2001. W. Jiang, J. Lennox, S. Narayanan, H. Schulzrinne, K. Singh and X. Wu, "Integrating Internet Telephony Services", IEEE Internet Computing (magazine), May/June 2002 (Vol. 6, No. 3). 9. K. Singh, Gautam Nair and H. Schulzrinne, "Centralized Conferencing using SIP", 2 nd IP-Telephony Workshop (IPTel'2001), April 2001. 10. K. Singh and H. Schulzrinne, "Unified Messaging using SIP and RTSP", IP Telecom Services Workshop 2000, Atlanta, Georgia, U. S. A, Sept 2000. K. Singh and H. Schulzrinne, "Unified Messaging using SIP and RTSP", Columbia University Technical Report CUCS-020 -00, NY, Oct 2000. 11. K. Singh, H. Schulzrinne, "Interworking Between SIP/SDP and H. 323", 1 st IP-Telephony Workshop (IPTel'2000), April 2000. K. Singh and H. Schulzrinne, "Interworking Between SIP/SDP and H. 323", Columbia University Technical Report CUCS-015 -00, NY, May 2000. 36