MTA Comparison

From Granizada

Jump to: navigation, search

Want to contribute? Send a patch to the wikitext (View Source tab above.)

This article was published at lwn.net in August, 2006 with many contributors on LWN and since then. Thanks to all.

Dan 10:12, 28 March 2007 (CST)


Contents

Mail Transfer Agents - Tool for the Job

For a lot of people the choice of the Mail Transfer Agent is important. A bad choice can mean lost time and money, lower reliability and increased risk to networks. A good choice can, depending on the architecture of your mail system, remain substantially unchanged for years. If you're designing a major mail system then you're also going to be asking yourself the question should I use more than one kind of MTA, and if so which should go where? Hopefully this article can help.

Too busy? Don't care about the pros and cons? Here is the Quick Answer.

I only cover MTAs related to Unix/Linux, and where these MTAs have leaked onto other operating systems. If you are planning to run a mail system on something else then you have specialist needs, or you have a problem, or both.

Debates over MTAs sometimes last for years, and this article covers the main points that come up over and over. Unfortunately, apart from this article there are no general comparisons of MTA characteristics on the Internet, and even very little benchmarking. The remarks here are personal opinions drawn from readily-verifiable facts and subjective comments drawn from experience. Nearly every MTA has a vociferous and sometimes combative group of supporters, not always including the principal authors of the MTA.

It is easy to see why administrators care about which MTA they use. Large installations require a lot of time spent tuning the MTA, and for any site email is commonly regarded as the most important use of the Internet.[1] End users can get by without a web site or a browser for a little, but without email business stops. And so countless administrators invest time in learning how to tweak their internet mail delivery tool in order to meet their various goals. But which tool should they use when?

As at the beginning of 2007, most Internet email seems [2] to be delivered by one of four MTAs:

  1. Postfix
  2. Exim
  3. qmail
  4. Sendmail

There are other worthy free MTAs to talk about, such as zmailer and smail3, but since they are not so widely used I omit them. There are some MTAs that are a small component of a larger system such as Courier-MTA (despite the name, which doesn't fit any definition of MTA used here!) which it isn't reasonable to compare. There are some unworthy MTAs too, these I am delighted to omit. There are also changes afoot I expect to cover here soon, including the rise of blackbox front-end email systems and new MTAs making their debut.

How To Compare MTAs

Current and recent versions (say, since 2002) of each of these four widely-used MTAs have broadly similar features. All of them have, on the basis of a very large body of evidence [3], the following characteristics:

  • a good security record (at least good; some have an excellent record)
  • ability to handle very large amounts of mail
  • interact with databases in many formats
  • can speak many of the SMTP variants in use
  • source code is available in a free manner
  • quality third-party documentation is available
  • there are significant user communities

They even have logos!

There are some assumptions implicit in the rest of this article. This document is not for you if you are looking for a product that:

  • address dozens or hundreds of other functions besides delivering mail. There's a fuzzy boundary, but none of the MTAs here were ever intended to be user-programmable groupware servers such as Lotus Notes or Microsoft Exchange!
  • presents a comprehensive GUI administrative interface as a major selling point. Such GUIs exist for most MTAs considered here, but they are by no means a selling point. If you're making a choice on the basis of the administrative GUI (as opposed to range of administrative functions, and how easily and efficiently they can be used) you probably need an email consultant.

Lotus Notes and Microsoft Exchange are examples of systems that don't meet the criteria for this document. They are able function as MTAs, but that is only a small part of what they were designed to do.

No MTA scores well by all measures. The needs of users vary greatly and some criteria are mutually orthogonal. Commonly cited MTA selection criteria are:

  • Ease of administration
  • Security
  • Performance
  • Long-term viability

Design features partly decide how much each MTA meets these criteria, with the rest coming from where the development team have interests. Contradictory examples of these features are:

  1. single configuration file, so everything is in one place
  2. many single-purpose and optional configuration files
  3. minimal and careful syntax
  4. powerful embedded scripting language
  5. maximum code stability
  6. source code contributions regularly incorporated
  7. minimum possible features added

These features are all valid. Which features you need depends on what you're looking for.

Just about every mail delivery scenario can be met, in one way or another, by all four MTAs. There is no one right answer.

Surveys

There is no equivalent of the Netcraft Web Survey for SMTP servers. SMTP surveying is a very different task to what Netcraft (and other companies) do for the web, but not particularly harder. It is astonishing how little definitive information there is about what SMTP servers are in use on the public Internet.

I picked the Big Four MTAs for this article based on discussions with the people who run busy mail hubs in public ISPs, by lurking in the email community and from taking small samples of what is running behind the world's MX records. I also take into account that many administrators do not change the MTA that comes with their Unix-like operating system. No major Linux distribution ship Sendmail by default, except for Red Hat prior to 2007[4], and only Sendmail on commercial Unix and some BSD platforms. Indicators such as the Debian Popularity Contest are flawed as statistical measures although they can give a flavour of what is happening in one corner of the Internet.

In January 2007 an article was published with the hopeful title Fingerprinting the World's Mail Servers and it illustrates that SMTP surveys are not so easy. Despite making what appears to be a very competent effort the authors settle for sampling 400k domains belonging to companies listed by a US data firm. While interesting, this data clearly cannot tell us which SMTP servers are run on the Internet. I have asked Ken Simpson and Stas Bekman if they would mind sharing their detection code for incorporation into a more complete survey design.

Dan Bernstein's 2001 SMTP survey of one million hosts put Sendmail at about 42% market share, but the methodology described is unlikely to give representative results for the Internet as a whole. Like the Simpson/Bekman article there is only an overview description of the approach, so it cannot be replicated.

qmail

qmail Summary
qmail-logo.png
MTA details
Website: http://www.qmail.org
Out since: 1996
Goals: Security; Simplicity; Efficiency
Non-goals: Unix conventions, ease of admin
Licence: not open source or free
Classification
Config: Many simple control files
Releases: Never (since 1997!)
Committers: 1
Maj. contribs: 0
Flexibility: Very, if you study hard
Subjective Comments
Administration: Buy one of the books!
Security: Good record
Performance: Excellent
Community: Smallish but very active
Sendmail compat: Good

Note: qmail is unmaintained: the author has not released since 1997 and does not permit others to make releases. It is also not Open Source Software, although the source is visible and usable within very tight restrictions.

So why should anyone care about qmail? Perhaps they shouldn't in 2007, but there were good reasons to notice qmail in its first five years of life, and there is still considerable publicity surrounding it, and some very happy users:

  • qmail had a radical and seemingly impregnable [5] security design.
  • qmail solved in one stroke all the problems of the hideous BSD mailbox format with the Maildir message format.
  • qmail was fast. If your only other choices were Sendmail or smail, qmail was a welcome alternative. In those days a lot of high-volume list management was still done on IBM mainframes; qmail was one of the earliest serious alternatives.

These days these advantages are at least equalled by other MTAs, and Maildir has become Maildir++. Yes, qmail taught MTA users and developers some lessons. No, qmail isn't reagarded as a realistic option these days: it doesn't support modern mail standards or even IPv6; it isn't maintained; it isn't possible for someone else to maintain it; its many oddities are increasingly painful relative to any benefits qmail has.

In 2007 a qmail maestro is someone who knows which collection of patches to apply to a ten year-old program. In 2004 a group of qmail experts put together netqmail, a distribution of qmail patches. But even that hasn't been touched for years, and now there are patches for the netqmail patch collection. Then there was qmailrocks which claims to be superceded (with a non-trivial upgrade path) by the author of the patches used by qmailrocks. Someone has pointed me to qmail-ldap as being used by ISPs. As observed at the beginning, you can use qmail to build a significant mail hub. But with the requirement to negotiate the chaos of qmail patch sets, you need to have a very special brand of dedication to ensure your mail hub is a good one. You might find qmail works for small sites without much configuration, but that will entirely depend on how your particular email workload is skewed -- for example, you might suddenly become invisible to some of your customers due to backscatter spam.

qmail is still used on some very high-volume sites, and there are still people who strongly believe that qmail is very correct code. qmail source is legal to copy, use and patch. The author's licencing analysis is thought-provoking but thus far irrelevant to the global copyright debate.

qmail comes with more free personality than nearly any other program -- what other MTA is likely to ask "hath the daemon spawn no fire?" when it can't start?

See qmail.org.

Postfix

Postfix Summary
postfix-logo.png
MTA details
Website: http://www.postfix.org
Out since: 1997
Goals: Security; Easy of use; Standards
Non-goals: General purpose MTA
Licence: IPL (a disused license)
Classification
Config: Single control file
Releases: Infrequent
Committers: 1
Maj. contribs: 3
Flexibility: Easy to change
Subjective Comments
Administration: Intermediate, good docs
Security: Good record. Credible team.
Performance: Excellent
Community: Medium-sized
Sendmail compat: Very good

Design goals: Secure, easy to administer, efficient.

Postfix is, like qmail, written by a prolific Unix security software author, this time Wietse Venema although unlike qmail the result is a suite with an interface like most other Unix programs. Postfix is almost entirely the work of Wietse, with occasional contributions in isolated areas such as integrating the Transport Layer Security (TLS) libraries. Releases come in bursts, with some releases containing only very small improvements. Release management is by Wietse personally.

Postfix has a monolithic main configuration file like Exim and Sendmail[6]. It is table-driven, everything is a table and a table can be represented in all kinds of ways from plain text files to databases to relational databases and more. It handles regular expressions in many contexts, using the Perl Compatible Regular Expression library developed by Phil Hazel for Exim. Postfix consists of about 150k lines of code.

Postfix fits somewhere between qmail and Exim. It consists of several programs (but fewer than qmail), and has a monolithic configuration file. Postfix has a strong emphasis on security, but not to the extent of imposing unusual Unix management practices. Postfix is quite flexible in its configuration file, but not to the extent of Exim. Postfix postdates qmail and follows a vaguely analogous security approach, an approach which was relatively more important in 1997. The Sendmail team explicitly acknowledge Postfix as an influence on their next-generation replacement for Sendmail, MeTA1.

Postfix is less versatile than Exim, and this is largely due to its foremost design criteria being security. Wietse specialises in writing software that is demonstrably harder to break, and that design goal prohibits some very convenient features that Exim can offer. Postfix is further towards the paranoia end of the security spectrum than Exim or Sendmail. The impressive achievement is that Postfix delivers such great flexibility and ease of administration despite meeting strict security goals. qmail is also regarded as highly secure, however it does not, by design policy, give the administrator anything like the flexibility of Postfix. If you have good reason to be paranoid, these sorts of considerations may mean that in a mail architecture you use Postfix for Internet-facing services and something else behind it.

Postfix has been measured by many as being extremely fast, and I have found it very efficient. My impression is that it is more efficient than Exim but not to a noticeable on modern hardware degree even with very high load. Postfix and qmail seem to use about the same amount of memory but by deliberate design qmail uses more bandwidth than Postfix because qmail only ever sends a single message per SMTP session even if there are multiple messages going via the same host. Postfix is quite Unix-centric with its secure design and is not maintained on very non-Unix platforms such as Windows.

Postfix is, like Exim, a drop-in replacement for Sendmail. Besides just implementing the sendmail command line interface, Postfix is compatible with Sendmail milters, an impressive and unequalled achievement to those that have investment in such modules.

The Postfix community is very active. Online documentation is quite good but scattered. There are three Postfix books in English.

See http://www.postfix.org and http://postfixwiki.org .


Sendmail

Sendmail Summary
sendmail-logo.png
MTA details
Website: http://www.sendmail.org
Out since: 1982
Goals: Be backwards-compatible
Non-goals: Best practice
Licence: Bespoke Open Source
Classification
Config: Single control file
Releases: Regular
Committers: many
Maj. contribs: many
Flexibility: Enormous, but complex
Subjective Comments
Administration: Hard to do well
Security: Terrible. Better now.
Performance: Ok for many
Community: Large
Sendmail compat: Bug for bug :-)

Design goals: Current Sendmail must be backwards-compatible. (The forthcoming MeTA1 project, formerly Sendmail X, is a rewrite and has design goals similar to Postfix.)

Sendmail 8 consists of about 118k lines of code, but that does not count the functionality in the M4 scripts used to generate the config file, nor milters. Documentation is good, and uniquely among MTAs there is a dominant company dedicated to Sendmail services. The Sendmail Consortium is dedicated to maintaining the Sendmail codebase.

Sendmail has an extraordinarily obscure configuration file, a poor history of security breaches and a design centred around Unix in the early 1980s. Sendmail is known for being inefficient compared to alternatives so it might be hard to see why sendmail is still used at all, but history has its own inertia. There is no good reason for a site without Sendmail experience to install it, given the effectiveness of the alternatives. There are often good reasons why a site with Sendmail working should keep it.

Despite all this, Sendmail:

  • has improved greatly in security and performance since about 2000, and has a large number of new features
  • is installed by default on most commercial Unix operating systems (but few Linux distributions)
  • works with little or no modification with the default settings
  • has a large following of systems administrators who have battled with and now understand to some extent how to configure and run it
  • is a well-known MTA name, see previous comment about inertia

Sendmail has been ported to many systems, including some that are not Unix-like such as Windows. Postfix isn't realistically portable to Windows, and Exim is something of a second-class citizen on Windows since it runs via Cygwin. So portability might be a reason to run Sendmail.


Exim

Exim Summary
exim-logo.png
MTA details
Website: http://www.exim.org
Out since: 1995
Goals: General purpose MTA
Non-goals: Security
Licence: GPL
Classification
Config: Single control file
Releases: Regular
Committers: 1
Maj. contribs: many
Flexibility: Enormous
Subjective Comments
Administration: Straightforward
Security: Quite good
Performance: Very good
Community: Large
Sendmail compat: Very good

Design goal: General-purpose MTA for Unix machines.

Exim was inspired by the author's work with the smail 3 sourcecode, which was itself provoked by the many problems of sendmail. So Exim too is a Sendmail drop-in replacement.

The outstanding feature of Exim is the intention that it be a general-purpose mailer. Exim is not a total rethink about how mail works, like qmail is. Nor does it restrict its feature set in order to achieve theoretical security, like Postfix. Exim instead tries to give administrators what they asked for, with of course a strong interest in security, reliability and performance. Exim cannot ever be considered as secure as Postfix, but then, it seems to be quite secure enough for most normal applications. Of the many compromised Unix machines I've met quite a few have been running Exim but there was no suspicion that Exim was the cause of the breach. Think about this carefully, a lot of people mistakenly view security as an absolute where it is instead a continuum[7]. In exchange for a lower level of design security -- for example, a willingness to cooperate with unknownable external programs for handling messages throughout message processing -- you get versatility Postfix cannot deliver. The counterpart to this paragraph is in the Postfix section. As a historical footnote, Exim3 had some flaws and consequently a couple of serious security breaches. Exim4 has a different design and has not had serious issues.

Exim behaves much like any traditional Unix daemon, with a monolithic configuration file, a monolithic daemon, small number of log files and a standard style of spooling. It has a very good security record over the last seven years (early releases had classic security issues), can cope with high load, and it has excellent integration facilities. Exim can be extended in many ways - it is even possible to compile in the entire perl interpreter to call from the configuration file! If there is an MTA feature, then Exim can support that feature in some way or another. Exim is very tightly specified and documented. Many features can be omitted at compile-time, making a special-purpose Exim easy to create. Exim has its own filter language, implementing the functionality that procmail most commonly use plus some extras.

Exim is used at some very high-volume sites where it provides good service, so performance comparisons saying qmail and Postfix are faster and handle queuing better don't necessarily have any bearing on real-world conditions (in 2007 on current hardware and with current definitions of high load.)


Afterwords

Using More than One MTA

A split MTA architecture is often useful. For example, separate MTAs for each of:

  • receiving mail from the Internet, and first level of anti-spam/malware
  • processing mail, ie second level of anti-spam/malware and site-specific processing
  • delivering mail locally, ie saving to whatever mailstore solution you use (Maildir, IMAP, etc)
  • sending mail to the Internet

There are efficiencies to using just one MTA, including in human resources and potentially reduced human error. Equally there can be good reasons to choose different MTAs for different purposes.

One scenario is to use Postfix and Exim. The logic for Postfix is that an MTA facing the Internet needs to be as secure as possible it must perform as few functions as possible. For Exim, an MTA used to maximise the quality of email in a second level of anti-spam/malware must be as flexible as possible.

Where there can be disc I/O performance problems, another scenario is to use just the delivery portion of Postfix for local deliveries because it is relatively small and fast, and Exim for everything else.

For sites already running Sendmail but experiencing performance problems it can help to leave all the routing logic to Sendmail (with the configuration files developed over many years) but to do the intensive spam/malware processing with either Exim or Postfix.

Open Source at Work

One of the interesting things about the three non-Sendmail MTAs here is the ideas and code that are shared. Postfix uses the Perl Compatible Regular Expressions library developed for Exim. Exim understands the Constant Database Format developed for qmail, and the Maildir mail file format also qmail. Postfix can use the Constant Database Format, and can use Sendmail Milters. And the successor to Sendmail explicitly acknowledges design inspiration from Postfix and qmail.

When Local Security Isn't a Problem

The reason - not the biggest but the only - reason why Unix MTAs have to work so hard at security is because of the Unix tradition of local delivery to files owned by an individual user. There is a kind of collective Unix racial memory relating MTAs to high-risk operations. Anything listening on port 25 must have root privilege because it is a privileged port, but all kinds of other servers have the same issue including web servers.

The MTA mixture of setuid binaries, specially-owned directories, pedantic authentication of local destinations, paranoia over filesystem access is all to do with having the MTA write to a file owned by some other user, usually by impersonating that user. Of course that is fraught with potential danger, and no matter how well the code is written a careless administrator can still make the entire process unsafe by changing file permissions. It can be difficult to impose modern partitioning security schemes on top of traditional Unix local delivery too.

But in millions of sites this is no longer an issue, because mail is kept in a central (usually IMAP-accessible) mailstore until the user chooses to view it. Mail comes into the SMTP daemon, which then makes an LMTP delivery to the IMAP daemon. Nothing touches local files. You can run the daemon as user nobody. It is possible to configure at least three of these mailers so that none of the potentially dangerous code is even on your system. Here's how:

  • Postfix.
  • Exim. All routers, directors, and transports are compiled only when specified in the Local/Makefile. You can compile Exim with only the smtp transport - and make that use LMTP to 127.0.0.1 for "local" delivery if you want. Then you can run Exim entirely in "unprivileged" mode, where it runs as "exim" the entire time, except when starting up the listening daemon.
  • qmail.

Quick Answer

The answer is Exim. If Exim doesn't do it for you then look at the pros and cons and don't read this section :-)

Exim can solve any MTA problem at least well and often excellently, has very good documentation and a most supportive community. It is the only modern mailer to expressly aim to be general-purpose. That is why it is always the best default recommendation. There are no ordinary circumstances where Exim is a bad choice, although there are circumstances where something else will clearly be better.

Think of Exim as the Linux of free MTAs. There are many free Operating Systems and some of them are better than Linux for specific tasks. But Linux can do (at least) a good job for nearly everyone[8].

This table is a bit less of a blanket statement:

MTA Suitability from 0 (bad) to 3 (good)
if you are... qmail Exim Sendmail Postfix Notes
Inexperienced 0 3 1 3 Exim and Postfix have good docs and clear examples
Worried about security 3 2 0 3 Postfix is secure and modern; qmail is secure but very old and cranky; Exim is secure to different criteria (read above.)
Relying on Sendmail milters 0 1 3 2 Postfix can run milters; can use equivalent Exim routers/filter script
Wanting minimum hassle 0 3 0 3 Sendmail has some easy front-ends, but the deeper you go the worse it gets. Postfix and Exim are more predictable.
Resource-constrained 3 2 1 2 See Embedded Application below for other comments
On Windows 0 2 3 0 Sendmail has a native Windows port; Exim is in the Cygwin distro
Needing commercial support 1 3 3 3 There are competent companies for all MTAs; qmail is inherently less supportable being so old

Embedded Applications

This article is not about embedded MTAs, mail servers running on tiny boxes, mobile phones etc. However several commentators have taken exception to my weightings under Resource Constrained. It isn't as simple as you might think, and these weightings can only be illustrative. Here are points to consider:

  • resources means more than memory. Sometimes having just one process in memory is better than several that get pulled in as needed, even if the total amount of memory used is greater, because questions arise such as "pulled in from where?" and "at what cost to do the pulling?". For a long-running embedded device with resource constraints that needs a full-featured MTA chances are Exim or Sendmail will be more suitable than Postfix or Sendmail. If you're using XIP (Execute In Place) this changes things again.
  • Embedded devices are designed according to specific criteria, not general ones. So you may be looking for a small and fast MTA, or a small and full-featured MTA, or an MTA that can boot very quickly. Postfix and qmail can be ready to receive before the rest of the system has even finished booting because they have a very minimal and separate receiving process that doesn't have to wait for other libraries to be initialised. These days a Unix computer can fit inside an RJ45 socket using a ColdFire CPU and at that level these considerations matter. (I did say this article doesn't cover embedded :-)
  • Exim can be stripped down by not including some features at build time. It still isn't tiny, but it certainly doesn't represent the code size you see if you use say size `which exim` on your Unix machine. On the other hand, if the task to be solved is very limited in scope, you may be able to use just a few programs from Postfix, probably smaller than the most stripped-down Exim you can build.
  • qmail has licensing problems. Read the fine print, but you'll probably need explicit permission to ship qmail in an appliance. Remember, it isn't Open Source or even close.

Some Home Truths

  • Sendmail can be made to do anything but is for people with a Sendmail background. It makes little sense for people who don't have a specific need for Sendmail to learn it. The beauty of this recommendation is that if everyone follows it, Sendmail will be dead in a generation. Good. The Sendmail developers have started again drawing from Postfix and qmail. Good again.
  • qmail is a specialist product with a lot of drawbacks in general use. qmail requires a very substantial committment to master. Unless you have a good reason to use it, don't. A hunch that qmail is more secure is not a good reason, for most normal purposes Postfix and Exim are just as secure. The usage terms (there isn't a licence, it is worth reading why) is a serious isuse for longevity considerations.
  • Postfix is very flexible, but is also is deliberately limited by its design [9] and has a tiny development community (not to be confused with its large user community.) So it has a less predictable future. The IBM licence has been abandoned by IBM, and precludes sharing with GPL code due to a GPL3-like view of software patents.
  • None of these points are personal opinion.

Footnotes

  1. Possibly changing fast since in 2007 many young people almost exclusively use instant messaging or similar communication models
  2. I.E. seems to me personally, which is why I have included a section on surveys that also talks about my subjective impressions!
  3. Evidence of the sort that says "here's a list of high-volume mail hubs running MTA x, so it can certainly be done", rather than a rigorous survey -- did I mention there is a section on surveys? This evidence can be seen on the mailing lists where professional mail admins gather. You'll find readily-verified postings from people talking about their scaling issues in very large organisations including ISPs and multinational companies. See for yourself (and if you want to do a survey based on that sort of information I'd might be able to help.)
  4. As of February 2007 there is evidence that the Fedora 7 test1 default MTA is Exim, and debate as to whether that is a good thing
  5. The Georgi Guninski vulnerabilities have been multiply confirmed, and one of them is a root vulnerability. DJB rejects this in typical style claiming it is an improbable configuration, but security analysis is always seeking improbable setups! DJB is wrong, but qmail still has an outstanding record.
  6. Actually Postfix also has a file called master.cf which is both somewhat obscure and very rarely touched. However, it does exist!
  7. As Bruce Schneir has explained in numerous articles and demonstrations
  8. Which doesn't stop me learning from the others -- thankyou NetBSD for ISBN 0-201-79940-5 and ISBN 0-321-16607-8 !
  9. That is, well-justified security design considerations. To take one example, Postfix has removed support for the @ character appearing in email usernames, and Wietse has stated he will never restore that ability. There are a lot of people who need this support, and Postfix cannot practically provide this, by design. Similarly there is much less flexibility for performing content scanning by external programs partway through SMTP processing, so there will never be an equivalent to Exim's Exiscan interface.
Creative Commons License
Creative Commons Attribution iconCreative Commons Share Alike icon
This content is licensed under the Creative Commons
Attribution ShareAlike License v. 2.5:
http://creativecommons.org/licenses/by-sa/2.5/
GNU head GFDL: Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". (shearer.org uses but does not currently recommend the GDFL and here's the explanation why. )
Personal tools
Navigation