Search


print PDF Ireland
Email Problems And Solutions

For dedicated servers, we have preinstalled server e-mail qmail

www.qmail.org

Qmail can be captured through the following patches:

  • patch tls permits the sending of e-mails with ssl cryptage

  • patch spamcontrol


Before Starting

The e-mail server is a program that accepts e-mails at port 25 via SMTP protocol. To find out more on how the SMTP protocol works, see the RFC: http://www.ietf.org/rfc/rfc0821.txt

Here is an example of SMTP communication with port 25:


$ telnet mx3.ovh.net 25 Resolving mx3.ovh.net

Trying 213.186.33.73...

Connected to mx3.ovh.net.

220 Welcome ! You are on server 8 called mx3.ovh.net, spam protected: www.spam-rbl.com ESMTP

HELO ping.ovh.net We introduce ourselves

250 Welcome ! You are on server 8 called mx3.ovh.net, spam protected: www.spam-rbl.com

MAIL FROM: We say who is sending the email

250 ok

RCPT TO: We say where we want to send the email to

250 ok acceptation

DATA

354 go ahead

test The email body. Usually you must enter FROM, TO, SUBJECT

.

250 ok 1065864990 qp 27338

QUIT Quit the connection to the server

221 Welcome ! You are on server 8 called mx3.ovh.net, spam protected: www.spam-rbl.com

Connection closed by foreign host.


The information in DATA must normally be the e-mail itself i.e. simply have the form of an e-mail:


From: oles@ovh.net

To: oles@ovh.net

Subject: test

test


Why was the e-mail accepted by the mx3.ovh.net. server?

- Because the IP of the connection ping.ovh.net (213.186.33.13) can relay the e-mails via mx3.ovh.net
- mx3.ovh.net is the MX of the ovh.net server

Relay

The simplest function of the e-mail server is the relay which:

  • accepts e-mails and then sends 250 ok after RCPT TO

  • places e-mails in the spool or the queue

  • resends e-mails to the MX server destination


It is an SMTP server that distributes e-mails on the internet.

The service provider access proposes this type of server. Any other type of SMTP server can also distribute e-mails.

The first step is presented in the Before Starting section. This simply consists of accepting e-mails. The problem becomes more complex with spam management which is presented in the following:

-the qmail-smtp process that does this function. Its role is to communicate in SMTP and save e-mails under a file directory on the hard disk.

The second step consists of stocking e-mails on the disks while waiting for the third step.

To see the spool status:

# /var/qmail/bin/qmail-qstat

queue

messages in queue: 360

messages in queue but not yet preprocessed: 0

This means that there are 360 e-mails that the server has already sent once visibly. So either the destination does not exist, or the MX server destination doesn't work correctly.

It should be pointed out that there are often problems with server relay response due to large amounts of emails in the spool.

Qmail can become enormous with more than 25,000 e-mails in the spool. In this case, the e-mail preparation becomes longer and longer and the messages in queue but not yet pre-processed increases.

To see the e-mails:

# /var/qmail/bin/qmail-qread | grep remote | grep -v done
remote dr_adams@sm66.com
remote 66-2434019-directway.com?amysummers@trailerr.trailblazernews.com
remote 2245031-5-reply@msgbnc.tgtrewards.net
remote response@sleekzine.com
remote larhondabeirne@hushmail.com

...

Why are these e-mails there ? These e-mails are there because:

  • the server relay has accepted the relay, but the RCPT TO destination does not exist

  • the server relay has accepted the relay, but the MX server destination doesn't exist

  • the MX server has accepted the email but the local POP3 account does not exist, so we try to send the email to the sender with the error but:

    • the recipient does not exist (MAIL FROM from the initial email)

    • the recipient MX server does not exist




qmail-send manages the queue for any qmail. It's role is to look at the files that qmail-smtp has left and to distribute. It has the choice of using a local distribution (in the case of the recipient, it is on the machine itself i.e. the server is the MX server of the email and there is a POP3/IMAP account on the machine) or remote (relay the email to the email recipient).

The third step is to send the email to the recipient. The email is then taken from the spool to find out who the recipient is. qmail searches the MX server of the domain and then connects to port 25 to send email. This operation is Cette opération est the same at the beginning.

We may have several MX in a domain with the same weight or different. In this case, the SMTP server first tries to connect to the MX server with a lowest weight. If two MX have the same weight, it is the one that comes out of the DNS query first that will be used.

For errors, see the error section.

Each relay operation is logged in the email headers.
We can see from where and through which mail server the email was sent:

Delivered-To: ovh.net-60gp@ovh.net

Received: (qmail 15496 invoked by uid 505); 11 Oct 2003 09:22:30 -0000

Received: from unknown (HELO 200-102-028-198.fnsce7005.dsl.brasiltelecom.net.br) (200.102.28.198)by mx4.ovh.net with SMTP; 11 Oct 2003 09:22:28 -0000

Reading it upwards from the bottom, we see that the email is sent from 200.102.28.198 which is connected to mx4.ovh.net and that the email arrived at the 60gp@ovh.net account.

See the forged headers in the Spam section

To see the functioning of your email server you can look in the logs. By default you only have the logs of the third step:

# tail -f /var/log/qmail/current

@400000003f87e06f3172d234 end msg 195408

@400000003f87e07005d90c24 delivery 3534821: deferral: Sorry,_I_wasn't_able_to_establish_an_SMTP_connection._(#4.4.1)/

@400000003f87e07005d91fac status: local 0/15 remote 1/120

@400000003f87e0740c22b274 new msg 195408

@400000003f87e0740c22be2c info msg 195408: bytes 1580 from qp 30987 uid 503

@400000003f87e0740c7fb6f4 starting delivery 3534823: msg 195408 to local hfifi-geefnie.com-studio@hiffi-gefenie.com

@400000003f87e0740c7fc694 status: local 1/15 remote 1/120

@400000003f87e0741c7b34d4 delivery 3534823: success: did_0+0+1/

@400000003f87e0741c7b4474 status: local 0/15 remote 1/120

@400000003f87e0741c7b485c end msg 195408


status: local 0/15 remote 1/120 on the spool->local distribution (the POP3 account) qmail uses 0 connections in 15 and on spool->remote (the third step) 1 in 120.

Attention: the file turns i.e. if it reaches a certain size it will redo the queue for the logs.
You also have the queue order -F that reopens the file when it turns

What Can Relay

To manage the relay functions it must handle the IPs of users who can use this feature. If you permit all IPs to relay, in time your machine will be used as a relay server for spammers.

To manage this you have a file called /etc/tcp.smtp

# cat /etc/tcp.smtp
127.0.0.:allow,RELAYCLIENT=" "
:allow

This means that:

  • all emails that come from IP 127.0.0.X are relayed

  • all email that come from other IPs are accepted only if the recipient is configured on the machine (see the MX server)



You can add an IP of your choice (e.g. if you have a fixed IP in your company or a fixed IP in adsl) and compile tcp.smtp:

# tcprules /etc/tcp.smtp.cdb /etc/tcp.smtp.tmp < /etc/tcp.smtp

Changes are made and implemented in the file /etc/tcp.smtp.cdb that are used by the SMTP server. You do not need to restart the SMTP server. The file reads each SMTP connection.

In the case of ISPs, the IP classes are being hard on the SMTP server that allows customers of the ISP to automatically use the SMTP relay server without any other configuration

The open-smtp or pop-smtp option consists of using the POP3/IMAP server to identify the POP3 server. By using a login and password you can be sure that it's someone you trust and thus enable relay. During the POP3/IMAP operation, the POP3 server registers the IP connection, updates /etc/tcp.smtp and compiles /etc/tcp.smtp.cdb

# cd /etc/vpopmail/
  1. ls -l
total 696
-rw-r-r- 1 vpopmail vchkpw 25 fév 24 2003 inc_deps
-rw-r-r- 1 vpopmail vchkpw 32 fév 24 2003 lib_deps
-rw-r-r- 1 root root 700229 oct 11 13:22 open-smtp
-rw-r-r- 1 root root 0 oct 11 13:22 open-smtp.lock

All the temporary IPs are stored in open-smtp:

# head -n 5 open-smtp
213.41.188.249:allow,RELAYCLIENT=" ",RBLSMTPD=" " 1029581749
213.41.184.98:allow,RELAYCLIENT=" ",RBLSMTPD=" " 1029782162
213.41.184.135:allow,RELAYCLIENT=" ",RBLSMTPD=" " 1029880909
213.41.186.107:allow,RELAYCLIENT=" ",RBLSMTPD=" " 1029960881
213.41.186.122:allow,RELAYCLIENT=" ",RBLSMTPD=" " 1030146813

The last figure is the time, in seconds, since 1970 and lets you know when the IP was used POP3/IMAP the last time. So this allows another script /home/vpopmail/bin/clearopensmtp to watch all the IPs older than 3 hours and to remove the /etc/tcp.smtp.

# cat /etc/crontab
...
40 * * * * root /home/vpopmail/bin/clearopensmtp 2>&1 >/dev/null

Warning: it must be used to indicate the user and then save

40 * * * * root /home/vpopmail/bin/clearopensmtp 2>&1 >/dev/null

and not

40 * * * * /home/vpopmail/bin/clearopensmtp 2>&1 >/dev/null

Moreover, in this case we see that this machine on the open-smtp file is particularly large.

# wc -l open-smtp
11859 open-smtp


11,859 IPs is a lot. And it's hard to know because there is no user specified in the crontab. We correct the error in crontab and we execute clearopensmtp:

# ls -l open-smtp
-rw-r--r-- 1 root root 700229 oct 11 13:29 open-smtp
  1. /home/vpopmail/bin/clearopensmtp
  2. ls -l open-smtp
-rw-r--r-- 1 vpopmail vchkpw 529 oct 11 13:29 open-smtp
  1. wc -l open-smtp
9 open-smtp

  1. cp /etc/crontab /etc/crontab.OLD
  2. pico /etc/crontab
  3. /etc/rc.d/init.d/crond restart
Arrêt de crond : OK
Démarrage de crond : OK
  1. diff -u /etc/crontab.OLD /etc/crontab
- /etc/crontab.OLD Sat Oct 11 13:31:15 2003
+ /etc/crontab Sat Oct 11 13:31:24 2003
42 4 1 * * root run-parts /etc/cron.monthly

  1. vpopmail
-40 * * * * /home/vpopmail/bin/clearopensmtp 2>&1 >/dev/null
+40 * * * * root /home/vpopmail/bin/clearopensmtp 2>&1 >/dev/null

It is better. The permissions for open-smtp have also changed. This is not very serious.

If you have the POP3/IMAP server busy with more than 3000-4000 connections/hour, we recommend using the OSD patch (Open-Smtp-Dir) you can find at
ftp://ftp.ovh.net/made-in-ovh/vpopmail. This patch was made by OVH and is to stop using the open-smtp file with the system locked (IPs blocked). If many people connect to the POP3/IMAP server, they must each wait their turn to have access to the file to save the IP as this slows the server. Instead we had the idea to use a directory to save files with the name of the IP and a size 0. Each new connection is used to update the file date and tmpwatch deletes files older than 3 hours. Another script updates /etc/tcp.smtp every second.

To avoid use pop3 before you can send emails, we can patch qmail with the smtp-auth patch. This patch allows you to send via SMTP connection the login and password for a POP3 account and then allow sending emails:

# telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
220 Welcome ! You are on server 1 called mx1.ovh.net, spam protected: www.spam-rbl.com ESMTP
EHLO ping.ovh.net
250-Welcome ! You are on server 1 called mx1.ovh.net, spam protected: www.spam-rbl.com
250-AUTH LOGIN CRAM-MD5 PLAIN
250-AUTH=LOGIN CRAM-MD5 PLAIN
250-PIPELINING
250-STARTTLS
250-8BITMIME
250 SIZE 10000000
AUTH LOGIN
334 VXNlcm5hbWU6
cG9zdG1gregreRlciVtbC5vdmgubmV0
334 UGFzc3dvcmQ6
errrob3Zo
235 ok, go ahead (#2.0.0)
MAIL FROM:
250 ok
RCPT TO:
250 ok
DATA
354 go ahead
truc
.

250 ok 1065758232 qp 5296
QUIT
221 Welcome ! You are on server 1 called mx1.ovh.net, spam protected: www.spam-rbl.com
Connection closed by foreign host.

We see that instead of HELO, we send EHLO which allows use of advanced ESMTP control described in http://www.ietf.org/rfc/rfc2821.txt. The server responds to EHLO with all the options that are accepted:

250-AUTH LOGIN CRAM-MD5 PLAIN it acts as smtp-auth
250-AUTH=LOGIN CRAM-MD5 PLAIN
250-PIPELINING define the type of dialogue
250-STARTTLS ssl or tls or encrypted connection
250-8BITMIME acceptance of email by 8bits and not by 7bit SMTP
250 SIZE 10000000 the maximum email size is 10 MB

The login and password are sent as base64. No penalty charge unlike chains sent to find out my passwords, they are false (the chains).

The installation of this patch is complex and requires a few tricks that are off-topic here. Please do not contact support for this. Instead, talk about it on the mailing-list.

You can also put in place the IP of the recipient's machine hardcoded in the configuration of qmail. This is very useful when, for example, the MX server in front is not very accessible (e.g. caramail.com). In this case, we can tell the server to use a special SMTP server whose role is to relay emails for caramail.com only.

# cat /var/qmail/control/smtproutes
caramail.com:213.186.33.9

Thus, all emails to caramail will be redirected to 213.186.33.9 that will handle the distribution of emails to caramail.com. Therefore, we avoid having a spool of important emails and we unload the machines. Of course, 213.186.33.9 must accept emails to go or we relay the emails but we trusts the admin to have properly configured servers. No need to restart the server after a change.

It is the qmail-remote process that responsible for connecting servers on port 25 and send emails.


MX Server


Emails eventually must get to the destination MX server i.e. one that will store the mail waiting to be consulted in POP3 server or IMAP server. In this case, the mail server accepts the email and did exactly the same as in the case of relay. It is the third step that changes.

In the third stage the qmail-sent process detects that the recipient exists on the machine. We remember it is the qmail-sent process that is involved in the second step to manage the spool. Then it starts to load up the lists of all the domains that are on the machine. These lists are in:

# ls -l /var/qmail/users/assign
-rw-r--r-- 1 root root 24546 oct 9 10:32 /var/qmail/users/assign

DO NOT change the files manually. They have a special structure and there is always a cdb file that must be compiled.

# head -n 4 /var/qmail/users/assign
+00-00l0-00l00l-0-0.com-:00-00l0-00l00l-0-0.com:508:503:/home/vpopmail/domains/00-00l0-00l00l-0-0.com:-:: +00h01.com-:00h01.com:508:503:/home/vpopmail/domains/00h01.com:-:: +01emarket.com-:01emarket.com:508:503:/home/vpopmail/domains/01emarket.com:-::

And so, if the recipient is found on the machine qmail-sent uses qmail-local to send the email locally on the proper directory. Inside it should find a .qmail-default file or it is vpopmail that takes charge of the distribution of the email with all the system redirections, alias, mailing list, catch all and the POP3 accounts with a quota management.


MX Configuration


You can get information in this guide: VpopmailAndAdministration


Qmail Parameters


Maximum file size

# echo "10000000" > /var/qmail/control/databytes

All emails that are larger than 10MB will be rejected.

Lifetime of an email in the spool

# echo "846000" > /var/qmail/control/queuelifetime

If the email remains in the spool for 10 days and is still not returned when it returns two errors to the postmaster account on the machine. If the file does not exist, the email is returned after five days.

Reject emails with certain MAIL FROM

# echo "spammeur@spam" > /var/qmail/control/badmailfrom

During the SMTP dialog, the SMTP server checks the MAIL FROM. We can refuse certain MAIL FROM which prevents the reception of some spam or viruses that have a fixed sender.

Attention: it has nothing to do with the From: To: Subject: fields of the email.


Errors


Sorry, I wasn't able to establish an SMTP connection

An attempt to send to dr_adams@sm66.com

We ask the DNS servers to find the MX:

# dig sm66.com mx

; <<>> DiG 9.2.1 <<>> sm66.com mx
; global options
printcmd

;; Got answer:
; ->>HEADER<<- opcode
QUERY, status: NOERROR, id: 32527

; flags
qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0


;; QUESTION SECTION:
;sm66.com. IN MX

;; ANSWER SECTION:
sm66.com. 86400 IN MX 10 mail.sm66.com.

; Query time
396 msec

; SERVER
127.0.0.1#53(127.0.0.1)

; WHEN
Sat Oct 11 12:09:00 2003

; MSG SIZE rcvd
47



If you use djbdns we can use dnsmx directly:

# dnsmx sm66.com
10 mail.sm66.com

So, the email server that manages all the @sm66.com emails is mail.sm66.com.
We will therefore try to send the email:

# telnet mail.sm66.com 25
Trying 66.239.204.101...
Connected to mail.sm66.com.
421 newd1.sm66.com out of connection slots
Connection closed by foreign host.

Ouch! Obviously the server does not work properly. We can see the error message ("out of connection slots"), which makes us think there are too many connections on the mail server at the moment. Qmail will try to resend the email later.

Another example: 66-2434019-directway.com?amysummers@trailerr.trailblazernews.com
Hmmm ... it looks like a spam email.

# dnsmx trailerr.trailblazernews.com
10 stdin-mv-01.trailblazernews.com
  1. telnet stdin-mv-01.trailblazernews.com 25
Trying 206.131.244.8...
telnet: connect to address 206.131.244.8: Connection refused

The MX server is present on the internet but there is no SMTP server that works and so we cannot connection on port 25.

# ping stdin-mv-01.trailblazernews.com
PING stdin-mv-01.trailblazernews.com (206.131.244.8) from xxxxxxx : 56(84) bytes of data.
64 bytes from news1mv.trailblazernews.com (206.131.244.8): icmp_seq=0 ttl=240 time=83.048 msec
64 bytes from news1mv.trailblazernews.com (206.131.244.8): icmp_seq=1 ttl=240 time=81.381 msec

stdin-mv-01.trailblazernews.com ping statistics

3 packets transmitted, 2 packets received, 33% packet loss
round-trip min/avg/max/mdev = 81.381/82.214/83.048/0.881 ms
  1. telnet stdin-mv-01.trailblazernews.com
Trying 206.131.244.8...
Connected to stdin-mv-01.trailblazernews.com.
Red Hat Linux release 7.2 (Enigma)
Kernel 2.4.7-10 on an i686
login:
Login incorrect

login:
Login incorrect

login:
Login incorrect

login:
Connection closed by foreign host.

Another example: response@sleekzine.com

# dnsmx sleekzine.com
20 mail2.sleekzine.com
30 mail3.sleekzine.com
10 mail1.sleekzine.com

There are three MX servers with different weights for each server. Therefore Qmail will try mail1.sleekzine.com which has the lowest weight, then mail2.sleekzine.com, then mail3.sleekzine.com.

# telnet mail1.sleekzine.com 25

Trying 216.93.189.135...

telnet: connect to address 216.93.189.135: Connexion terminée par expiration du délai d'attente

  1. telnet mail1.sleekzine.com 25

Trying 216.93.189.135...

telnet: connect to address 216.93.189.135: Connexion terminée par expiration du délai d'attente

  1. telnet mail3.sleekzine.com 25

Trying 216.93.189.133...

telnet: connect to address 216.93.189.133: Connexion refusée

If we look at what will make qmail with an email like that:

# echo "test" | mail -s "test" response@sleekzine.com

  1. ps auxw

...

qmailr 7066 0.0 0.5 2568 736 ? S 15:04 0:00 qmail-remote sleekzine.com root@nsxxxx.ovh.net response@sleekzine.com

...

  1. tail -f /var/log/qmail/current

@400000003f880003024f7034 new msg 66208

@400000003f880003024fbe54 info msg 66208: bytes 229 from qp 7065 uid 0

@400000003f88000303f530f4 starting delivery 67496: msg 66208 to remote response@sleekzine.com

@400000003f88000303f5447c status: local 0/10 remote 1/20

@400000003f88003f221f1d74 delivery 67496: deferral: Sorry,_I_wasn't_able_to_establish_an_SMTP_connection._(#4.4.1)/

@400000003f88003f221f30fc status: local 0/10 remote 0/20


553 sorry, that domain isn't allowed to be relayed thru this MTA

If the server does not relay the email, this means that:

  • the IP connection is not in /etc/tcp.smtp (see the Relay section)

  • the recipient does not have an account on the machine (see the MX section)


$ telnet mx1.ovh.net 25

Trying 213.186.33.29...

Connected to mx1.ovh.net.

220 Welcome ! You are on server 2 called mx1.ovh.net, spam protected: www.spam-rbl.com ESMTP

HELO ping.ovh.net

250 Welcome ! You are on server 2 called mx1.ovh.net, spam protected: www.spam-rbl.com

MAIL FROM:

250 ok

RCPT TO:

553 sorry, that domain isn't allowed to be relayed thru this MTA (#5.7.1)

QUIT

221 Welcome ! You are on server 2 called mx1.ovh.net, spam protected: www.spam-rbl.com

Connection closed by foreign host.


In this case one can easily guess that the account wanadoo.fr is not managed on mx1.ovh.net and it is simply a relay problem: the IP connection is not declared in /etc/tcp.smtp (voir section relay)

If the account is declared on the email server and you have this error, verify that the domain is ok in /var/qmail/bin/assign and if you have restarted qmail-send with

# killall -HUP qmail-send


This may be due to a false declaration on the DNS server when you specify a mail server that has nothing to do with your domain and therefore you do not know and who did not want to relay your emails .

qmail_has_prog_delivery_but_has_x_bit_set._(#4.7.0)

This means that the /home/vpopmail directory has execution permissions but it should not be the case (chmod 700 instead of chmod 600 in a .qmail-alias)

Here is a script to correct the permissions:

#!/bin/sh

cd /home/vpopmail/domains

chown -R vpopmail.vchkpw .

for file in `find . `

do

if -d $file

then

chmod 700 $file

else

chmod 600 $file

fi

done


553 sorry, that domain isn't in my list of allowed rcpthosts (#5.7.1)

This simply means that your domain is not configured in /var/qmail/control/rcpthosts

Exemple the domain 'test.com' has been installed but erased accidentally from /var/qmail/control/rcpthosts
Here's what you get when we try to send an email to postmaster@test.com :

$ telnet nsXXXX.ovh.net 25

Trying 213.186.XX.XX...

Connected to nsXXXX.ovh.net.

220 nsXXXX.ovh.net ESMTP

HELO ping.ovh.net

250 nsXXXX.ovh.net

MAIL FROM:

250 ok

RCPT TO:

553 sorry, that domain isn't allowed to be relayed thru this MTA (#5.7.1)

QUIT

Connection closed by foreign host

$


Repeat the Queue Directory

See the guide : QueueQmailFull

Spam

The Spam section is coming soon.