Electronic Mail Electronic Mail Most heavily

Electronic Mail

Electronic Mail • Most heavily used application on any network • Uses client-server architecture – Electronic mail client accepts mail from user and delivers to server on destination computer – Many variations and styles of delivery • Simple Mail Transfer Protocol (SMTP) – TCP/IP – Delivery of simple text messages • Multi-purpose Internet Mail Extension (MIME) – Delivery of other types of data – Voice, images, video clips

SMTP Concept

User Agents and Mail Transfer Agents

Relay MTAs

Mail Gateways

The Entire E-mail System

SMTP • Simple Mail Transfer Protocol is standard application protocol for delivery of mail from source to destination • Built on top of TCP: provides reliable delivery of data • SMTP does not normally use intermediate mail servers for sending mail • SMTP uses persistent connections • Other functions: – E-mail address lookup – E-mail address verification • Example protocol exchange

Sample SMTP interaction C: S: C: S: C: C: C: S: c: Telnet hamburger. edu 25 220 hamburger. edu HELO ce. sharif. edu 250 Hello ce. sharif. edu, pleased to meet you MAIL FROM: 250 ali@ce. sharif. edu. . . Sender ok RCPT TO: 250 bob@hamburger. edu. . . Recipient ok DATA 354 Enter mail, end with ". " on a line by itself Do you like ketchup? How about pickles? . 250 Message accepted for delivery QUIT 221 hamburger. edu closing connection

SMTP • RFC 821 • Not concerned with format of messages or data – Covered in RFC 822 (see later) • SMTP uses info written on envelope of mail – Message header • Does not look at contents – Message body • Except: – Standardize message character set to 7 bit ASCII

Basic Operation • Mail created by user agent program (mail client) – Message consists of: • Header containing recipient’s address and other info • Body containing user data • Messages queued and sent as input to SMTP sender program – Typically a server process (daemon on UNIX)

Mail Message Contents • Each queued message has: – Message text • RFC 822 header with message envelope and list of recipients • Message body, composed by user – A list of mail destinations • • Derived by user agent from header May be listed in header May require expansion of mailing lists May need replacement of mnemonic names with mailbox names • If BCCs indicated, user agent needs to prepare correct message format

SMTP Sender • Takes message from queue • Transmits to proper destination host – Via SMTP transaction – Over one or more TCP connections to port 25 • Host may have multiple senders active • When delivery complete, sender deletes destination from list for that message • When all destinations processed, message is deleted

Optimization • If message destined for multiple users on a given host, it is sent only once – Delivery to users handled at destination host • If multiple messages ready for given host, a single TCP connection can be used – Saves overhead of setting up and dropping connection

Possible Errors • • Host unreachable Host out of operation TCP connection fail during transfer Sender can re-queue mail – Give up after a period • Faulty destination address – – User error Target user changed address Redirect if possible Inform user if not

SMTP Protocol - Reliability • Used to transfer messages from sender to receiver over TCP connection • Attempts to provide reliable service • No guarantee to recover lost messages • No end to end acknowledgement to originator • Error indication delivery not guaranteed • Generally considered reliable

SMTP Receiver • Accepts arriving message • Places in user mailbox or copies to outgoing queue forwarding • Receiver must: – Verify local mail destinations – Deal with errors • Transmission • Lack of disk space • Sender responsible for message until receiver confirm complete transfer – Indicates mail has arrived at host, not user

SMTP Forwarding • Mostly direct transfer from sender host to receiver host • May go through intermediate machine via forwarding capability – Sender can specify route – Target user may have moved

Conversation • SMTP limited to conversation between sender and receiver • Main function is to transfer messages • Rest of mail handling beyond scope of SMTP – May differ between systems

Figure 3. 9 SMTP Mail Flow

SMTP System Overview • Commands and responses between sender and receiver • Initiative with sender – Establishes TCP connection • • Sender sends commands to receiver e. g. HELO Each command generates exactly one reply e. g. 250 requested mail action ok; completed

SMTP Replies • Leading digit indicates category – – Positive completion reply (2 xx) Positive intermediate reply (3 xx) Transient negative completion reply (4 xx) Permanent negative completion reply (5 xx)

Operation Phases • Connection setup • Exchange of command-response pairs • Connection termination

Connection Setup • Sender opens TCP connection with receiver • Once connected, receiver identifies itself – 220 service ready • Sender identifies itself – HELO • Receiver accepts sender’s identification – 250 OK • If mail service not available, step 2 above becomes: – 421 service not available

Mail Transfer • Sender may send one or more messages to receiver • MAIL command identifies originator – Gives reverse path to used for error reporting – Receiver returns 250 OK or appropriate fail/error message • One or more RCPT commands identifies recipients for the message – Separate reply for each recipient • DATA command transfers message text – End of message indicated by line containing just period (. )

Closing Connection • • Two steps Sender sends QUIT and waits for reply Then initiate TCP close operation Receiver initiates TCP close after sending reply to QUIT

Electronic Mailboxes and Addresses • E-mail users have an electronic mailbox into which mail is deposited • Usually associated with computer account; one user may have different electronic mailboxes • User accesses mail with a mail reader program • Electronic mailbox is identified by an e-mail address mailbox@computer

Internet Mail Addressing • User portion or mailbox portion is site specific – Joe – Seyyed_hamid_1221 – 1234. 5678 • Host portion or computer portion is domain name • Source mail client – Resolves destination name using DNS – Contacts mail delivery server at destination – Copies mail to server • Destination mail server – Interprets user name according to local mailbox addresses – Places mail in appropriate mailbox

E-mail message format • Simple two-part format – Header includes delivery information – Body carries text of message • Header and body separated by blank line

E-mail Headers • Lines of text in format keyword: information • keyword identifies information; information can appear in any order • Essential information: – To: list of recipients – From: sender – Cc: list of copy recipients • Useful information – Reply-to: different address than From: – Received-by: for debugging • Frivolous information: – Favorite drink: lemonade – Phase-of-the-moon: gibbous

Data in E-mail • Original Internet mail carried only 7 -bit ASCII data • Couldn’t contain arbitrary binary values; e. g. , executable program • Techniques for encoding binary data allowed transport of binary data • Uuencode: three 8 -bit binary values as four ASCII characters (6 bits each) – Also carries file name and protection information – Incurs 33% overhead – Requires manual intervention

Multipurpose Internet Mail Extension (MIME) • Extension to RFC 822 • SMTP can not transmit executables – Uuencode and other schemes are available • Not standardized • Can not transmit text including international characters (e. g. â, å, ä, è, é, ê, ë) – Need 8 bit ASCII • Servers may reject mail over certain size • Translation between ASCII and EBCDIC not standard • SMTP gateways to X. 400 can not handle none text data in X. 400 messages • Some SMTP implementations do not adhere to standard – CRLF, truncate or wrap long lines, removal of white space, etc.

MIME • Multipart Internet Mail Extensions - Extends and automates encoding mechanisms • Allows inclusion of separate components – programs, pictures, audio clips – in a single mail message • Sending program identifies the components so receiving program can automatically extract and inform mail recipient – Header includes MIME-Version: 1. 0 Content-Type: Multipart/Mixed; Boundary=Mime_separator – Separator line gives information about specific encoding Content-Type: text/plain

MIME (cont. ) • MIME is extensible – sender and receiver agree on encoding scheme • MIME is compatible with existing mail systems – Everything encoded as ASCII – Headers and separators ignored by non-MIME mail systems • MIME encapsulates binary data in ASCII mail envelope

MIME

Overview of MIME • Five new message header fields – – – MIME version Content type Content transfer encoding Content Id Content Description • Number of content formats defines • Transfer encoding defined

Content Types • Text body • Multipart – Mixed, Parallel, Alternative, Digest • Message – RFC 822, Partial, External-body • Image – jpeg, gif • Video – mpeg • Audio – Basic • Application – Postscript – octet stream

MIME Transfer Encodings • Reliable delivery across wide largest range of environments • Content transfer encoding field – Six values – Three (7 bit, 8 bit, binary) no encoding done • Provide info about nature of data • Quoted-printable – Data largely printable ASCII characters – Non-printing characters represented by hex code • Base 64 – Maps arbitrary binary input onto printable output • X-token – Named nonstandard encoding

Figure 3. 10 Printable Encoding of Binary Data into Radix-64 Format

Programs as mail recipients • Can arrange for e-mailbox to be associated with a program rather than a user’s mail reader • Incoming mail automatically processed as input to program • Example – mailing list subscription administration • Can be used to implement client-server processing – Client request in incoming mail message – Server response in returned mail reply

Mailing Lists and Forwarders • E-mail addresses can be attached to programs as well as electronic mailboxes • Mail exploder or mail forwarder resends copies of message to e-mail addresses in mailing list – UNIX mail program sendmail provides mail aliases – Mailing list processor, e. g. listserv, can also interpret subscription management commands

Mail Gateways • Mailing list processing may take significant resources in large organizations • May be segregated to a dedicated server computer: mail gateway – Provides single mail destination point for all incoming mail – Can use MX records in DNS to cause all mail to be delivered to gateway

Mail gateways and forwarding • Users within an organization may want to read mail on local or departmental computer • Can arrange to have mail forwarded from mail gateway • Message now makes multiple hops for delivery • Hops may be recorded in header • Forwarded mail may use proprietary (non-SMTP) mail system

Mail Gateways and E-mail Addresses • Organization may want to use uniform naming for external mail • Internally, may be delivered to many different systems with different naming conventions • Mail gateways can translate e-mail addresses

Mailbox Access • Where should mailbox be located? • Users want to access mail from most commonly used computer • Cannot always use desktop computer as mail server – Not always running – Requires multitasking OS – Requires local disk storage • Can TELNET to remote computer with mail server

Mail Access Protocols • Typically users do NOT run a server on their local PC • User agent accesses a mailbox from a shared mail server • Mail access protocol transfers mail from server to local PC • 2 popular protocols: – POP 3 (Post Office Protocol v 3) – IMAP (Internet Mail Access Protocol) • Can’t use SMTP because getting messages is a pull operation & SMTP is a push protocol.

Mail Access Protocol • Instead of TELNEt, use protocol that accesses mail on remote computer directly • TCP/IP suite includes Post Office Protocol (POP) for remote mailbox access – Computer with mailboxes runs POP server – User runs POP client on local computer – POP client can access and retrieve messages from mailbox – Requires authentication (password) – Local computer uses SMTP for outgoing mail

Email Protocols

POP 3 and SMTP

POP 3 • Three phases – Authorization – Transaction – Update • During a POP 3 session, the server maintains state information • POP 3 downloads mail messages to the local machine – not good for “nomadic user”

Dialup Access and POP • POP useful for dialup connection – User’s computer not always connected – Can download all mail at once and read off-line – Can compose mail off-line and mail in one connection

POP 3 protocol authorization phase • client commands: – user: declare username – pass: password • server responses – +OK – -ERR transaction phase, client: • list: list message numbers • retr: retrieve message by number • dele: delete • quit S: C: S: +OK POP 3 server ready user bob +OK pass hungry +OK user successfully logged C: S: S: S: C: C: S: list 1 498 2 912. retr 1 . dele 1 retr 2 . dele 2 quit +OK POP 3 server signing off on

POP 3 • Stands for Post Office Protocol – Version 3 • RFC in 1993 • POP 3 “is intended to permit a workstation to dynamically access a maildrop on a server host in a useful fashion. Usually, this means that the POP 3 protocol is used to allow a workstation to retrieve mail that the server is holding for it. “

POP 3 • Doesn’t provide extensive mail operations on the server • Normally mail is downloaded, then deleted • Deletion doesn’t really occur until QUIT • Uses port 110

Minimal POP 3 Commands • • • USER name – user’s name PASS string – user’s password QUIT – quits, and updates the mailbox STAT – status LIST [msg] – lists the mail messages (#’s only) RETR msg – retrieves a mail message DELE msg – deletes a message NOOP – No operation RSET – resets the mail to its original state

Example POP 3 Transaction Server

Establish the Socket Primary Thread Server Socket (110) Primary Thread Client

Server is OK +OK POP 3 server ready Primary Thread Server Socket (110) Primary Thread Client

Establish the Socket USER bob Primary Thread Server Socket (110) Primary Thread Client

This user is OK +OK Primary Thread Server Socket (110) Primary Thread Client

Send the Password PASS ITSD 4304 Primary Thread Server Socket (110) Primary Thread Client

The Password is OK +OK Primary Thread Server Socket (110) Primary Thread Client

Ask how the Server is Feeling… STAT Primary Thread Server Socket (110) Primary Thread Client

I’m Fine, Thanks! +OK 2 320 Primary Thread Server Socket (110) Primary Thread Client

Got Any Mail? LIST Primary Thread Server Socket (110) Primary Thread Client

Yup… +OK 2 messages (320 octets) Primary Thread Server Socket (110) Primary Thread Client

Message One 1 120 Primary Thread Server Socket (110) Primary Thread Client

Message Two 2 200 Primary Thread Server Socket (110) Primary Thread Client

End . Primary Thread Server Socket (110) Primary Thread Client

Get Message One RETR 1 Primary Thread Server Socket (110) Primary Thread Client

Message One Coming! +OK 120 octets Primary Thread Server Socket (110) Primary Thread Client

Message One Coming! Primary Thread Server Socket (110) Primary Thread Client

End of Message 1 . Primary Thread Server Socket (110) Primary Thread Client

Get Message One DELE 1 Primary Thread Server Socket (110) Primary Thread Client

Message 1 Gone! +OK message 1 deleted Primary Thread Socket (110) Primary Thread Client Server Note: it’s not really gone…

Get Message One QUIT Primary Thread Server Socket (110) Primary Thread Client

QUIT – Commits/Updates +OK bob POP 3 server signing off Primary Thread Socket (110) Primary Thread Client Server Note: now it’s gone

Socket Disposed Primary Thread Server Primary Thread Client

IMAP • Internet Mail Access Protocol • Allows users to manipulate remote mailboxes • Must maintain info about user (e. g. , folder hierarchy) even after the user quits • IMAP server is always in 1 of 4 states: – – Nonauthenticated Authenticated Selected Logout

POP 3 (more) and IMAP More about POP 3 • Previous example uses “download and delete” mode. • Bob cannot re-read e-mail if he changes client • “Download-and-keep”: copies of messages on different clients • POP 3 is stateless across sessions IMAP • Keep all messages in one place: the server • Allows user to organize messages in folders • IMAP keeps user state across sessions: – names of folders and mappings between message IDs and folder name

Security • SMTP, POP 3 and IMAP are plain text protocols • Secured version available to protect the user – SMTPS – POP 3 s – Need to be configured both on server and client

Stunnel • Name of an application/service – Even if the mail server application does not implement secure version • What does it do? – gathers the encrypted message from client – Decrypts the data – Sends plain text data using a local connection to insecure port • Server application considers it a normal client

STunneling Send decrypted data to port 25 of the server Stunnel Encrypted data (Command/mail body) (on server) Listening to port 995 Server Client

Web interface • Nowadays there are many web interface user agents • They allow accessing messages from any client with only a web browser • However the interface is not as sophisticated as special mail agents • But it is very useful if you need to check your emails from different computers

Summary for Email • Electronic mail allows quick, asynchronous communication across entire Internet • Can attach e-mail addresses to programs for processing – Mailing lists – Other client-server applications • Simple Mail Transfer Protocol (SMTP) is Internet standard for mail delivery • Mail gateways – Provide uniform user addressing outside organizations – Translate from Internet mail (e. g. SMTP) to proprietary systems • Post Office Protocol (POP) and Internet Mail Access Protocol (IMAP) allow remote access to electronic mailboxes.