Simple Mail Transfer Protocol (SMTP)

One important achievement was the publishing of the Mail Transfer Protocol (MTP), which was first defined in RFC 772 in September 1980, then updated in RFC 780 in May 1981. MTP describes a set of commands and procedures by which two devices can connect using TCP to exchange e-mail messages. Its operation is described largely using elements borrowed from two early TCP/IP application protocols that were already in use at that time: Telnet and FTP. The commands of MTP are in fact based directly on those of FTP.

There wasn’t anything inherently wrong with basing e-mail delivery on something like FTP, but defining it this way made MTP somewhat of a “hack”. It was also restricted to the capabilities defined by FTP, a general file transfer protocol, so it was not possible to include features in the protocol that were specific to sending and receiving mail. Due to the importance of e-mail, a specific protocol designed for the purpose of delivering e-mail was warranted. This protocol was first defined in RFC 788, published in November 1981: the Simple Mail Transfer Protocol (SMTP).

The name suggests that SMTP is “simpler” than the “non-simple” MTP that it replaced. Whether this is true or not is somewhat a matter of opinion; I do note that RFC 788 is 61 pages long, while the earlier RFC 780 was only 43 pages. What SMTP definitely has over MTP is elegance; the protocol is designed specifically for the transport of electronic mail. While it retains certain similarities to FTP, it is an “independent” protocol running over TCP. So, from a conceptual standpoint, it can be considered simpler than MTP. In terms of mechanics, the process SMTP uses to transfer an e-mail message is indeed rather simple, especially compared to some other protocols.

RFC 788 described the operation of SMTP carrying e-mail messages corresponding to the ARPAnet text message standard as described in RFC 733. Development of both e-mail messages and the SMTP protocol continued, of course. In August 1982, a milestone in TCP/IP e-mail was achieved when RFCs 821 and 822 were published. RFC 821 revised SMTP, and became the defining standard for the protocol for the next two decades. RFC 822, its companion standard, became the standard for TCP/IP electronic mail messages carried by SMTP.

 

As the 1980s progressed and TCP/IP and the Internet both grew in popularity, SMTP gradually overtook other methods to become the dominant method of e-mail message delivery. For a number of years, the protocol was used mostly “as is”, with no new RFCs published to define new versions or formally change its behavior.

 

This changed in February 1993, when RFC 1425, SMTP Service Extensions, was published. As the name suggests, this standard describes a process for adding new capabilities to extend how SMTP works, while maintaining backward-compatibility with existing systems. SMTP with these extensions is sometimes called Extended SMTP or ESMTP (though use of this term seems to be not entirely universal). As a development of SMTP continued, RFC 1425 was revised in RFC 1651 in July 1994 and then RFC 1869 in November 1995. Along with these, a number of other RFCs defining particular SMTP extensions such as pipelining and message size declaration were defined.

 

In April 2001, another major milestone in TCP/IP e-mail was reached when revisions of RFC 821 and RFC 822 were published, as RFCs 2821 and 2822 respectively. Both documents are “consolidations” of updates and changes that had been made to RFCs 821 and 822 between 1982 and 2001. And no, I don’t think it is a coincidence that the old and new RFC numbers are exactly “2000” apart. RFCs 2820 and 2823 were both published in May 2000, so it looks like 2821 and 2822 were reserved for the e-mail standards. I think this was a great idea, as it makes more clear that the new RFCs are revisions of the old ones.

 

RFC 2821 is the current base standard for SMTP. It incorporates the base protocol description from RFC 821, and the latest SMTP extensions as defined in RFC 1869. Perhaps more importantly, it updates the description of the e-mail communication model to reflect the realities of modern TCP/IP networks, especially the e-mail features built into the Domain Name System (DNS). We’ll examine this in more detail in the next topic.

 

The TCP/IP electronic mail communication model describes the way e-mail messages are conveyed from the sender to the recipient. In most cases, this involves the sender’s client machine sending the e-mail to its local SMTP server, which in turn sends it to the recipient’s local SMTP server, and finally to the recipient’s local host. All of these steps except for the last one are performed by SMTP. In fact, the overall e-mail communication model is largely described by the RFC 821 and 2821 SMTP standards.

 

The initial communication takes place between the sender’s client machine and a local SMTP server that the sender is allowed to access. After submission of the e-mail message, that SMTP server becomes responsible for delivering the message to the SMTP server responsible for the recipient’s mailbox. There are two different ways that this can happen.

 

In the first years of electronic mail, when RFC 821 and its predecessors were initially defined, the Internet was very different than it is today. There was no Domain Name System, and this made electronic mail delivery complex because there was no way to map a mailbox address to the IP address of the SMTP server that managed that mailbox. Also, there were many proprietary networks connected to the Internet, which meant that it was no always possible for any given system to communicate with any other.

 

Given this, how could e-mail be delivered? The most common way in the early days of SMTP was through a process called relaying. SMTP routing information was included along with the e-mail address, to specify a sequence of SMTP servers that the mail should be relayed through to get to its destination. For example, if a sender using SMTP server A wanted to send an e-mail to someone whose mailbox was on SMTP server Z, they might have needed to specify that the mail is sent through intermediate SMTP servers at sites D, P and U to get there. An SMTP connection would be established from A to D to send the message on one leg of its journey; then it would go from D to P, P to U and then U to Z. The process is analogous to how IP routing works, but at the application layer (actually using IP routing at a lower level, of course.)

 

 

You can probably see the problems with this quite easily: it’s cumbersome, requires many devices to “handle” the mail, results in delays in communication, and also requires the communication of source routes between SMTP servers. It was certainly functional, but far from ideal.

 

The creation of DNS radically changed how e-mail delivery worked. DNS includes support for a special mail exchanger (MX) record that allows easy mapping from the domain name in an e-mail address to the IP address of the SMTP server that handles mail for that domain.

 

 In the new system, SMTP communication is much simpler and more direct. The sending SMTP server uses DNS to find the MX record of the domain to which the e-mail is addressed. This gives the sender the DNS name of the recipient’s SMTP server. This is resolved to an IP address, and a connection can be made directly from the sender’s SMTP server to the recipient’s to deliver the e-mail. While SMTP still supports relaying, direct email delivery using MX records is faster and more efficient, and RFC 2821 makes clear that this is now the preferred method.

 

In this new system, SMTP is generally only used for two transfers: first, from the sender’s client machine to the sender’s local SMTP server, and then from that server to the recipient’s local SMTP server, as shown in Figure. (A distinct mail access protocol or method is used by the recipient for the last leg of the journey.) Each transfer of an e-mail message between SMTP servers involves the establishment of a TCP connection and then the transfer of the e-mail headers and body using the SMTP mail transfer process. The next two topics describe in detail how this occurs.

 

The delivery of electronic mail using the Simple Mail Transfer Protocol (SMTP) involves the regular exchange of e-mail messages between SMTP servers. SMTP servers are responsible for sending the e-mail that users to the server submit for delivery. They also receive the e-mail either intended for local recipients or in some cases for forwarding or relaying to other servers.

 

All SMTP communication is done using the TCP. This allows SMTP servers to make use of TCP’s many features that ensure efficient and reliable communication. SMTP servers generally must be kept running and connected to the Internet 24 hours a day, seven days a week, to ensure that mail can be delivered at any time. (This is a big reason why most end users employ access protocols such as POP3 to access their received e-mail rather than running their own SMTP servers.) The server listens continuously on the SMTP server port, well-known port number 25, for any TCP connection requests from other SMTP servers.

 

An SMTP server that wishes to send e-mail normally begins with a DNS lookup of the MX record corresponding to the domain name of the intended recipient’s e-mail address, to get the name of the appropriate SMTP server. This name is then resolved to an IP address; for efficiency, this IP address is often included as an Additional record in the response to the MX request, to save the sending server from needing to perform two explicit DNS resolutions.

 

The SMTP sender then establishes an SMTP session with the SMTP receiver. Once the session is established, mail transactions can be performed, to allow mail to be sent between the devices. When the SMTP sender is done, it terminates the connection. All of these processes involve specific exchanges of commands and replies, which are illustrated in Figure.

 

Let’s take a look at these processes in more detail, starting of course with SMTP session establishment. The SMTP sender begins by initiating a TCP connection to the SMTP receiver. The sending SMTP server uses an ephemeral port number since it is playing the role of the client in the transaction. Assuming that the server is willing to accept a connection, it will indicate that it is ready to receive instructions from the client by sending reply code 220. This is called the “greeting” or “service ready” response. It commonly includes the full domain name of the server machine, the version of the SMTP server software it is running, and possibly other information.

 

Now, it would be rude for the server acting as a client to just start sending commands to the responding server without saying hello first, wouldn’t it? So that’s exactly what comes next: the client says “hello”. In the original SMTP protocol, this is done by issuing a HELO command, which includes the domain name of the sending (client) SMTP server as a courtesy. The receiving device then responds back with a return “hello” message using an SMTP reply code 250.

 

When the sending device is finished sending all the e-mail it has to transfer to the receiving device, and done with all other activities, it terminates the session by issuing the QUIT command. This normally results in a 221 “goodbye” message from the SMTP receiver, saying something like “closing transmission channel”. The TCP connection is then terminated.

 

A server may also terminate prematurely in special cases. If it is given a local command to shut down (for example, due to imminent rebooting of the hardware server upon which it is running), it may respond to any routine command with a 421 response (“Service not available, closing transmission channel”). A server is not supposed to terminate a session simply due to receipt of an invalid command, however, only in special cases where session termination cannot be avoided.