Skip to the content of the web site.

Unix Sendmail and Secure Authentication

03-Aug-2006/15-Sep-2006

Most email clients (e.g. Outlook, Outlook Express, Eudora etc.) can be configured for secure communications using authentication and SSL (Secure Socket Layer) -- see the advice on Email Security. Users cannot use these protected services if the system manager does not provide them.

This note addresses the issue of securing sendmail(8) services on Unix/Linux systems so that end users may submit mail for delivery without exposing their userid and password in the clear.

Beware: this note does not address configuration issues for server applications used by clients to read their mail-- i.e., "pop" and "imap" services. Further, I have no knowledge of other mail servers like Postfix and Microsoft Exchange and can provide no advice on them.

Sendmail SSL Support

Your sendmail will support SSL (for encryption and privacy) and authentication if built with "STARTTLS" and "SASL" options. To confirm that try: 

[3:18pm minus] /usr/lib/sendmail -d0.1 -bv
Version 8.13.8
 Compiled with: DNSMAP HESIOD HES_GETMAILHOST LDAPMAP LOG
                MATCHGECOS MILTER MIME7TO8 MIME8TO7 NAMED_BIND
                NEWDB PIPELINING SASLv2 SCANF STARTTLS
                ...etc.
If you don't have support for both you should upgrade. IST supports an "xhier" version for recent versions of Solaris. Most Unix/Linux distributions include support -- we use Redhat but others have good experiences with Debian, FreeBSD and others. If you are running an older system where there is no support you should replace your system or move your mail services.

You will need an SSL key pair -- a certificate and private key for your host. If you have a key pair that you use to secure web services on the same system that will work -- but only if the host names used for web services and email services match. If you do not have a key pair you should approach the UW/IST Certificate Authority (IST-CA).

You should avoid using self-signed certs, these will confuse the end user. Many client applications will fail miserably when presented with a self-signed cert.

Ports and SSL

Virtually all mail servers require the client application (e.g., Outlook Express, Thunderbird, Eudora, etc.) to "login" to send mail -- that helps to prevent spam and forgeries. The login can be done on port 25 (smtp), port 465 (smtps) or port 587 (submission). On ports 25 and 587 the login is optionally protected by SSL encryption -- the client issues a "starttls" command to start the encryption. Port 465 requires the SSL encryption, it is not optional -- "smtps" on port 465 is like "https" on port 443. Outlook Express does not support SSL on port 587 and we have been recommending port 465 (smtps) where SSL is required for all clients. You may have clients using port 587, that's fine provided that you require SSL before authentication.

Mail servers requiring user authentication should be configured with certificates so they can do SSL to protect the userid and password. Users should configure their mail-client to use SSL. Finally mail servers should be configured to require SSL when users authenticate. If not done properly some users will expose their userid/password in the clear.

Xhier Sendmail-8.13

If you have a current xhier provided sendmail you need the SSL certificate key pair installed (if you don't have a key pair you can request one) and the certificate for the CA (Certificate Authority who signed your cert) in these locations: 

[3:42pm mino] grep Certs /etc/mail/sendmail.cf
O CACertPath=/software/sslCerts-1/config/certs
O CACertFile=/software/sslCerts-1/config/certs/cacert.pem
O ServerCertFile=/software/sslCerts-1/config/certs/smtp.pem
O ServerKeyFile=/software/sslCerts-1/config/certs/private/smtpkey.pem
O ClientCertFile=/software/sslCerts-1/config/certs/smtpc.pem
O ClientKeyFile=/software/sslCerts-1/config/certs/private/smtpckey.pem
The server and client key pairs are copies of the same data provided by the IST-CA -- the certificates are "smtp.pem" and "smtpc.pem", the private keys are "smtpkey.pem" and "smtpckey.pm".

The "cacert.pem" file is the certificate for the Certificate Authority that "issued" your certificates -- the IST-CA has the CA certificate(s) used by Thawte in their list of Thawte Signed X509.3 Certicates. Look for the "Issuer:" text in your certificate

The "cert" files are public documents which can be stored as world readable files: 

[3:52pm mino] cd /software/sslCerts/config/certs/
[3:55pm mino] ls -la cacert.pem smtp.pem smtpc.pem
   8 -r--r--r--   1 root     none        3130 Aug  1  1996 cacert.pem
   8 -r--r--r--   2 root     none        3534 Apr  6 15:03 smtp.pem
   8 -r--r--r--   2 root     none        3534 Apr  6 15:03 smtpc.pem
The "key" files are private documents containing a secret that should be protected and only readable by user "root". The directory and contents should be like this: 
[3:55pm mino] suw
Password:
[3:56pm mino]# ls -lad private
   2 drwx------   2 root     none         512 Apr 27  2004 private/
[3:56pm mino]# cd private
[3:56pm mino]# ls -la smtpkey.pem smtpckey.pem 
   2 -r--------   3 root     none         891 Dec 11  2000 smtpckey.pem
   2 -r--------   3 root     none         891 Dec 11  2000 smtpkey.pem

If you have these files in place then you need these configuration settings for xhier sendmail: 

[3:35pm mino] cd /software/sendmail/config/local
[3:35pm mino] more options
.....etc
smtp_auth=yes
use_smtps=yes
secure_auth=yes
# secure_auth=no
The "smtp_auth" setting configures sendmail to allow those who have authenticated to send mail through the server -- that should be "yes". The "use_smtps" setting configures the smtps service on port 465 -- the service we recommend your clients use. Again, that should be "yes". The "secure_auth" setting configures whether or not you allow clear text authentication on ports 25 and 587 -- if you have existing users you will want to set that to "no" for a migration period while your users configure their applications to use your secure services. When your users have updated their configuration set it to "yes".

Then you need to rebuild the sendmail configuration and restart the server on the new configuration: 

[3:43pm mino]# cd /software/sendmail/maintenance
[3:43pm mino]# ./sendmail-rebuild
[3:43pm mino]# ./sendmail-restart
If you see any gripes when rebuilding the configuration or restarting the daemon you will need to check your work carefully. If you need any assistance you should contact the package maintainer or file an RT item asking for assistance (I can provide assistance as well).

The xhier sendmail-8.13 package will bring with it the cyrus-sasl daemon -- that's an authentication daemon used by sendmail.

Linux/Unix Sendmail

Most Linux/Unix vendors provide a version of sendmail with support for SSL -- some do not. The following discussion takes RedHat Fedora as an example but the discussion should apply equally to other Linux/Unix systems including OS/X and FreeBSD.

First, make sure you have a version of sendmail that supports "STARTTLS" and "SASL" (see above). You will require a SSL key pair (a certificate and private key) from the IST-CA -- if you have a key pair that you use for web services that will work fine. You may require the certificate for the authority who issues your certificate. You should use a key pair from a well known authority -- avoid self-signed certificates.

Determining where your vendor wants to find the certificates and private keys can be a struggle. Try this strategy to determine where they should be: 

[3:27pm minus] cd /etc/mail
[3:27pm minus] egrep 'KEY|CERT' sendmail.mc
dnl define(`confCACERT_PATH',`/etc/pki/tls/certs')dnl
dnl define(`confCACERT',`/etc/pki/tls/certs/ca-bundle.crt')dnl
dnl define(`confSERVER_CERT',`/etc/pki/tls/certs/sendmail.pem')dnl
dnl define(`confSERVER_KEY',`/etc/pki/tls/certs/sendmail.pem')dnl
In this example the directory CACERT_PATH where certificates and keys are found is "/etc/pki/tls/certs" -- other vendors may use a different location. The vendor provided CACERT, "ca-bundle.crt", is a list of certs for known authorities -- make sure it contains the cert for the CA that issued your cert. This configuration stores the private key SERVER_KEY and certificate SERVER_CERT together in one file "sendmail.pem" -- other vendors may store them in separate files. Some vendors omit the CLIENT_CERT and CLIENT_KEY, you may see them like this: 
dnl define(`confCLIENT_CERT',`/etc/pki/tls/certs/sendmail.pem')dnl
dnl define(`confCLIENT_KEY',`/etc/pki/tls/certs/sendmail.pem')dnl

Find where the vendor wants certificates and private key and locate accordingly. Make sure that files containing the private key are well protected.

Edit "/etc/mail/sendmail.mc" to remove the "dnl" prefix on the lines identified above, add lines for "CLIENT_CERT" and "CLIENT_KEY" if your vendor hasn't provided them. That key pair is used when mail servers talk to each other.

To enable authentication remove the "dnl" prefix from these lines in "/etc/mail/sendmail.mc" -- 

define(`confAUTH_OPTIONS', `A p')dnl
...
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL GSSAPI DIGEST-MD5 ...
...
DAEMON_OPTIONS(`Port=submission, Name=MSA, M=Ea')dnl
Authentication requires the saslauthd(8) service -- you will need that installed and running.

To enable the "smtps" service remote the "dnl" prefix from this line in "/etc/mail/sendmail.mc" -- 

DAEMON_OPTIONS(`Port=smtps, Name=TLSMTA, M=s')dnl

Now you can generate a new configuration based on the settings you have configured. Try: 

[3:18pm minus]# cd /etc/mail
[3:18pm minus]# cp sendmail.cf sendmail.cf.orig
[3:18pm minus]# m4 sendmail.mc > sendmail.cf
Then you need to kill and restart the sendmail daemon 
[3:19pm minus]# service sendmail restart
Shutting down sendmail:                         [  OK  ]
Shutting down sm-client:                        [  OK  ]
Starting sendmail:                              [  OK  ]
Starting sm-client:                             [  OK  ]
If all goes well you should be ready for testing.

Testing

If you have successfully configured your system to support SSL and authentication services as above then these are some tests you might try to confirm your work.

If your tests fail and you need assistance you can seek support from IST. We have staff who support sendmail on many Unix platforms -- I can provide some assistance as well.

Some Observations

It should be clear that configuring sendmail services is non-trivial. There are a lot of components and it's easy to make a mistake. My best advice therefore is to minimize the number of servers you require. It is better to have a very few departmental (or faculty) mail servers than to have multiple servers. Mail servers on individual workstations is bad idea.

Within IST we have many Unix systems but only a very few of those are mail servers that accept and relay mail for clients. Most of our Unix systems are configured to punt all mail to another server -- they do not offer any of the services at ports 25, 465 and 587.

See also

For more information see also:

Finally, if you have any questions/concerns or need help please let us know.

IST Security Operation Centre