Messages from postfox/postdrop filling up space on server...

SUSE Linux Enterprise Server SLES Configure-Administer
1

Hi Guys,

We are using:
Novell Access Manager - Access Gateway Appliance 3.1.2 (i586)
VERSION = 3.1
PATCHLEVEL = 2

Issue:
Recently we have started getting the following error messages:
postfix/postdrop[29837]: warning: mail_queue_enter: create file maildrop/133977.29837: No space left on device

Above error messages are filling up in /var/log/ directory in a file called: warn-20140603
The error messages are continuos and have filled up entire partition taking down the partition to a lower disk space state.

Troubleshooting:
Removed file warn-20140603, but its gets generated again.

Any idea what could be the reason for postdrop to fillup logs like this?
Any pointers to tackle this issue?

Thank you,
-ddgaikwad

2

Hi ddgaikwad,

[QUOTE=ddgaikwad;21880]Hi Guys,

We are using:
Novell Access Manager - Access Gateway Appliance 3.1.2 (i586)
VERSION = 3.1
PATCHLEVEL = 2

Issue:
Recently we have started getting the following error messages:
postfix/postdrop[29837]: warning: mail_queue_enter: create file maildrop/133977.29837: No space left on device

Above error messages are filling up in /var/log/ directory in a file called: warn-20140603
The error messages are continuos and have filled up entire partition taking down the partition to a lower disk space state.

Troubleshooting:
Removed file warn-20140603, but its gets generated again.

Any idea what could be the reason for postdrop to fillup logs like this?
Any pointers to tackle this issue?

Thank you,
-ddgaikwad[/QUOTE]

well, how much space is available in the file system used for maildrop? With a standard config, that directory is /var/spool/postfix/maildrop…

There may be several non-standard situations that can cause this (a “zip bomb” being one), but usually this boils down to not having split off /var/spool/postfix into a separate file system and some other stuff in i.e. /var/spool, /var/log or even /home (if everything is in one common file system) filling up the space.

I strongly recommend to separate those things into separate file systems that you cannot directly control - user-writable directories (/tmp, /home, but various subdirs of /var/spool, too), /var/log and especially those “accessible” from the outside (Postfix spool and IMAP/POP3 mail stores).

Regards,
Jens

3

Hi Jens,

I have checked for the space on directories, and have about 8GB of file system size allocated to it.
Also, I saw that, when I kill all the postdrop, I see that they spring up again all over with sometimes going to about 100 processes.

What could be causing this issue?
Also, this issue started all of a sudden, and this is happening on our Dev servers, what would be repercussions, if this happens on our production servers?

-ddgaikwad

4

Hi ddgaikwad,

			[QUOTE=ddgaikwad;21918]Hi Jens,

I have checked for the space on directories, and have about 8GB of file system size allocated to it.
Also, I saw that, when I kill all the postdrop, I see that they spring up again all over with sometimes going to about 100 processes.

What could be causing this issue?
Also, this issue started all of a sudden, and this is happening on our Dev servers, what would be repercussions, if this happens on our production servers?

-ddgaikwad[/QUOTE]

since these processes all belong to the Postfix server, chances are high that you’re facing a lot of incoming messages. Does “postqueue -p” report a lot of messages?

If it’s actually a full queue with lots of processing that is killing your server, you might want to put all messages “on hold” (using “postsuper -h ALL”) and then check/delete/release messages one by one (“man postsuper” should give you an idea of how to do this).

What does sound a bit strange is that you save you have 8GB - is that actual free space in the file system carrying the Postfix spool space, or just the over-all size (which says absolutely nothing…)?

what would be repercussions, if this happens on our production servers?

Same as with the dev server: Too much to do, insufficient disk space, server busy. But most of all, it depends on how this was caused. If it’s actually a message flood, then you need to take counter-actions by limiting resources for Postfix (ensuring that a “loaded” Postfix doesn’t bring down all of the system → #1: separate file systems, #2: proper postfix limits configuration). If those messages are coming from internal sources, you may as well try to fix the real cause, but when the mail server is open to external messages, there’s not much you can do when you’re flooded, but limiting the effect on the remainder of the system…

Regards,
Jens

5

Hi Jens,

[QUOTE=jmozdzen;21927]Hi ddgaikwad,

since these processes all belong to the Postfix server, chances are high that you’re facing a lot of incoming messages. Does “postqueue -p” report a lot of messages?

If it’s actually a full queue with lots of processing that is killing your server, you might want to put all messages “on hold” (using “postsuper -h ALL”) and then check/delete/release messages one by one (“man postsuper” should give you an idea of how to do this).

What does sound a bit strange is that you save you have 8GB - is that actual free space in the file system carrying the Postfix spool space, or just the over-all size (which says absolutely nothing…)?

what would be repercussions, if this happens on our production servers?

Same as with the dev server: Too much to do, insufficient disk space, server busy. But most of all, it depends on how this was caused. If it’s actually a message flood, then you need to take counter-actions by limiting resources for Postfix (ensuring that a “loaded” Postfix doesn’t bring down all of the system → #1: separate file systems, #2: proper postfix limits configuration). If those messages are coming from internal sources, you may as well try to fix the real cause, but when the mail server is open to external messages, there’s not much you can do when you’re flooded, but limiting the effect on the remainder of the system…

Regards,
Jens[/QUOTE]

Its funny though, for one of the changes we had to restart our server and after the restart the server came up fine and we are no longer seeing multiple processes of post drop or filling up of log files and consumption of disk space…

I am still not sure what caused it… Well, I might need to move the servers to supported patch level and take this with an SR. :smiley:

thank you,
ddgaikwad