POP goes the server
How to go about choosing a POP service for your Unix e-mail server
May 1999
![[Next story]](/sunworldonline/icons/tb-next.gif){kind=icon}
![[Table of Contents]](/sunworldonline/icons/tb-toc.gif){kind=icon}
![[Search]](/sunworldonline/icons/tb-search.gif){kind=icon}
|
|
POP goes the serverHow to go about choosing a POP service for your Unix e-mail server
|
May 1999
|
Which POP's for you? Better yet, what server software should you use to deliver Post Office Protocol (POP) services? SunWorld columnists Cameron Laird and Kathryn Soraiz discuss the factors you'll need to consider when deciding which POP server is right for you. (2,000 words)
 Mail this article to a friend |
n March, SunWorld published a feature article we wrote on Postfix,
a Simple Mail Transfer Protocol (SMTP) that aims to compete with the
industry standard, sendmail (see Resources for
pointers to this article and related references). The volume of reader
response surprised us. More people than we realized are seeking
help in selecting mail servers.
To serve that need, Cameron has compiled a set of brief descriptions of the most popular choices for SMTP and POP servers (see Resources). This month, we return to SunWorld to look in more detail at the POP alternatives.
E-mail fundamentals
Remember, first, how most Internet e-mail works. A correspondent sits
at her desktop, composes a message, then selects Send or the
equivalent command. This step passes the text of the message to the
SMTP server she uses to reach the outside world. That server decodes
her list of addresses and relays the message on to your
e-mail server. There, the message is filed in a message store, often
called a spool, where it waits for your request to retrieve it.
If you're an old-time Unix hand, you might retrieve the message from its spool with a command-line utility like mailx, pine, ucbmail, or mush, hosted on the same machine as the message store. More frequent nowadays, though, is to have a desktop machine running Microsoft Win* or Mac OS periodically poll your e-mail server through the POP protocol, and then move any messages it finds from the server to your desktop, where you read them.
Mail transfer agents (MTAs) mediate the server-to-server communications. For two decades, the majority of MTAs have been sendmail running over Unix, but the competition has been heating up as alternative applications (including Postfix, qmail, CommuniGate Pro, and others) and operating systems (often Windows NT, but occasionally AS/400 or other special-purpose OSs) take on the standard configuration.
MTAs often share their "local" work with mail delivery agents (MDAs). An MDA is a process that places a message where a user can access it with a mail user agent (MUA). The most popular MUAs now are POP clients such as Eudora, Outlook Express, and Netscape Communicator. The focus of this article is on the MDAs that serve POP requests for these end-user applications. Here, we look at the POP servers in isolation from their clients. It's a recognized ideal of Web browsing that a Netscape browser reads documents served by a Microsoft server, and vice versa. One marker of POP's success as a protocol is that the same interchangeability is expected and achieved in reading e-mail.
Figure 1 below illustrates the physical elements and computing processes involved in e-mail transmission.
Figure 1. E-mail transmission |
|
|
|
|
|
What's the diff'?
MTA rivalry is recognized. Several commercial companies promote one or
another MTA, and conversation among system administrators on mailing
lists and at professional meetings often turns to the relative merits
of exim, Smail, Post.Office, and the rest of the SMTP tribe.
POP service receives an order of magnitude less attention. Most system administrators we know use whatever MDA was first installed on their e-mail server, and never update it. In part, this is because POP is far simpler, and therefore easier to get right, than SMTP. The entire source code for several viable POP servers is under a thousand lines.
However, as e-mail services continue to specialize, consequential differences between the leading POP servers are becoming apparent. When you review your own needs, you're likely to evaluate these dimensions of POP service:
Let's look at each of these factors in more detail.
Compatibility between MTA and MDA(s)
In principle, MTAs and MDAs are coupled only loosely. An MTA puts
messages in a message store, and a POP server retrieves them from the
message store, to pass on to the end user's MUA. At a minimum, then,
the two servers must recognize a common message store.
The traditional and still most common message store on Unix hosts is a flat file, one for each account, all in the same directory (the spool), each of which holds all the messages a particular account has received and not yet deleted. Essentially all POP servers have this capability.
This simplest kind of message store scales poorly. It's certainly adequate for departments with a dozen or a hundred accounts. With good hardware, it's practical to run a thousand or even ten thousand light-duty e-mail accounts using traditional spools.
What constrains this approach, though, is that most Unix filesystems don't do well with large numbers of names in a single directory. To handle a hundred thousand accounts (a modest number in an online world that already has several hundreds of millions of distinct e-mail addresses), a couple of resources are generally considered first.
Some vendors support a "hashed" filesystem, which manages very full directories gracefully. Such a change in the filesystem allows e-mail servers to be used without modification of their source or configuration.
Next comes the generalization of spool locations. A conventional spool for one of our accounts might be /usr/spool/mail/claird; directing that instead to /usr/spool/mail/c/claird, with analogous changes for other spools, generally provides adequate performance for a multiplied user base.
To manage million-account servers, as well as those with stringent performance requirements or advanced searching and other capabilities, the message store can be a relational database management system (RDBMS) or other high-end DMS. While several projects have hacked such capabilities into the popular MTAs and MDAs, several commercial packages, including CommuniGate Pro and the Netscape Messaging Server (NMS), offer them "out of the box."
If source-code hacking is an option for you, almost any of the POP servers (POPpers) can be regarded as compatible with any MTA, because the protocol is simple enough to permit relatively easy rewrites of the POP server. For out-of-the-box operation, the general recommendation is to use the POPper favored by your MTA provider or OS vendor: qmail-popup for qmail; Qpopper for most sendmail and sendmail-like (Postfix, exim, and so on) installations; and the supplied POP daemon for such commercial installations as Post.Office, CommuniGate Pro, NMS, and Sun Internet Mail Server.
Security
MTAs have a notorious security history. The most famous Internet worm
before Melissa, for example, was a sendmail exploit that memorably
paralyzed thousands of installations a decade ago. One response to
sendmail's difficulties has been the development of such open source
MTAs as qmail and Postfix, explicitly designed to answer security
concerns. In a different direction, commercial products like
CommuniGate Pro provide a highly integrated suite of tasks that
promise to keep all their consequences within their scope of
integration, and away from other host resources.
By comparison, the security profile of MDAs is often neglected. Several factors contribute to this oversight:
Because MDA security breaches are more likely to involve a break-in at one site than an infection of several, we suspect there's a tendency to keep POP exploits quiet and not admit to them. Experts know, though, that POPpers can be vulnerable, and alerts have occasionally been posted against Qpopper, ipop3d, and other widely used daemons.
Along with attacks through a POPper, the other main security concern about POP is that its remote use often involves sending passwords "in the clear."
What can you do about security? For protection of passwords, choose a POPper such as Qpopper that supports authenticated POP (APOP -- we've occasionally heard this glossed as "advanced POP"). Of course, your users will need clients that also support APOP -- Netscape Navigator 4.0 does not, but Eudora and Outlook Express do.
POPpers with a Berkeley heritage, including Qpopper, run as root during all operations. However carefully they've been reviewed and patched, this sort of design makes security experts very, very wary. qmail-popup and a few of the commercial e-mail packages have a more robust architecture, and qmail's designer, Daniel Bernstein, has a record of achievement in security.
Reliability
We've found no significant differences in reliability between the
leading POPpers, neither in our experience nor those of the system
administrators we've interviewed. Reliability is so important to us,
though, that we continue to research it, and will report useful results
on the "POP comparison" page listed in the Resources section.
Customizability
Most POPpers offer little customizability. To repeat an
insider joke about sendmail, "its natural scripting language is C."
On the other hand, the open-sourced POPpers are simple enough that tweaking logging functions or adding a hook for a POP-before-SMTP authorization (a common customization designed to discourage abuse by spammers) requires no deep expertise.
Commercial POP servers generally offer at least enough customizability to serve alternate port assignments and message stores. The first of these is particularly convenient when shaking down new or enhanced software; it's possible to run more than one POPper simultaneously on a single host, so that even live tests can be run on a production host without a traumatic transition.
Performance
Performance is a contrast to reliability: it's fairly easy to identify
faster and slower POP servers. Commercial POPpers generally are
swifter (sometimes very much so) than the open source alternatives.
In our experience, though, POPper performance is secondary. Almost without exception, MTAs or other MDAs weigh down the e-mail server more than POP does. If a POP daemon consumes enough cycles to matter, it's very, very likely its associated MTA demands far more, and any optimizations are best initiated by examining the MTA first.
Making a decision
In a few circumstances, POP isn't even the right kind of MDA for
delivery to users' desktops. The Internet Message Access Protocol
(IMAP), for example, is a well-established standard that appeared,
back in 1995, to be overtaking POP. It has stumbled since then,
for business reasons.
IMAP offers all POP's capabilities, plus more flexibility, holding out the hope that users can make more sophisticated use of the e-mail service. IMAP can be useful in a "controlled" environment, such as some college or corporate departments. For Internet service providers (ISPs), though, the overriding reality is that IMAP moves more processing onto servers and away from clients, and there's simply no place for those economic pressures in the current competitive ISP market. As consultant Russ Nelson of Crynwr Software told us, "Client CPU cycles are cheap and largely unused. Server CPU cycles are expensive and don't scale." Even in many internal business environments, it's habitual to boost functionality by rolling out more potent client-side machines. IMAP seems to prosper only in centrally organized computing cultures.
Other transports are even rarer among MDAs. Web-based services that rely on the HyperText Transport Protocol (HTTP) are in use, but generally don't perform well. The WebDAV extension to HTTP looks as though it might open new horizons for MDA and other initiatives, but it's still at least several months away from commercial release.
So how do you choose a POP server? First, take the advice of your MTA supplier, because MTA considerations invariably dominate server performance, reliability, security, and functionality. Confirm that you're current with your POPper's security patch list. Establish a solid baseline for any comparisons, then you can begin to experiment with different POP daemons and configurations. Vanilla installations, and organizations most comfortable with vendor-supplied software, generally stick with Qpopper. If security concerns you (and on today's hostile Net, it should), look into qmail-popup and commercial packages like CommuniGate Pro, Post.Office, and NMS. If you want to tinker at a source-code level, Qpopper, ipop3d, and several others are available under relaxed licenses. Large installations will likely use a commercial product, or a well-tuned Zmailer or qmail, with their associated POPpers.
All the POP daemons we've tested have had relatively easy installations. Most commercial vendors have a try-before-you-buy option.
Choosing a POP server needn't be traumatic. All the leading
alternatives are reasonably easy to understand. The range of worthy
alternatives means that, whatever the scale, security restrictions,
flavor of Unix, or required customizations you need, you're likely to
find at least one that's an adequate fit.
Thanks to Doug Hughes and Vladimir Butenko for their helpful discussion.
|
|
Resources
About the author
Cameron Laird and Kathryn Soraiz manage their own software consultancy, Network Engineered Solutions, from just outside Houston, TX. They write SunWorld's biweekly Regular Expressions column.
|
|
|
|
![[(c) Copyright Web Publishing Inc., and IDG Communication company]](/sunworldonline/icons/b-copyright99.gif){kind=icon}
If you have technical problems with this magazine, contact webmaster@sunworld.com
URL: http://www.sunworld.com/swol-05-1999/swol-05-pop.html
Last modified: