SMTP Monitoring probes fill up Postfix maillog
 

Hello there

I have a single server CentOS 7 with Postfix installed.
Just yesterday I adjusted our separate monitoring server to start probing the SMTP port every 30 sec. The probing works as it should and we will now get outage infos in case 3 probes fail in a row.

The problem with this is, that the Postfix maillog now logs every 30 sec the following 3 lines:

Apr 6 13:17:46 mail postfix/smtpd[4981]: connect from unknown[10.12.34.56]
Apr 6 13:17:46 mail postfix/smtpd[4981]: lost connection after EHLO from unknown[10.12.34.56]
Apr 6 13:17:46 mail postfix/smtpd[4981]: disconnect from unknown[10.12.34.56]

This makes reading the logfile fairly difficult and increases its size unnecessarily.

How can I get rid of those entries? Can I somewhere whitelist that specific host so it does not generate any log entries? I would also be happy if there would be an regexp or pcre solution in order to delete those entries on their way to the maillog.

As a last resort, I would also accept a solution which cleans the maillog let's say every hour from those entries, but I really hope there is a better solution...

Any help is greatly appreciated :-)

Cheers
Juri

Juri:
Welcome to LQ!
Tell us more about this probing, in detail, if possible.
What does the "monitoring server" consist of, or what does it do?

Sounds very syslog-like. Are you forwarding logs from the smtp host to the "monitoring server", or perhaps just
a simple test using netcat or telnet type utilities (this "probe")?

As a note from an experienced SysAdmin:
Scared of you.
As a Rule, I never touch logs in this manner.

What is this "probe" and what is this "monitoring server"?

Thanks.

Hi Habitual

Thanks for your assistance, much appreciated :-)

The monitoring server is a separate Windows box which runs the monitoring software The Dude v4 from Mikrotik.
The Dude is probing every 30 sec a separate CentOS 7 server running Postfix. The probe consists of sending an EHLO to the Postfix server and if a reply is received, the connection gets aborted.

The logfile in question is the standard Postfix logfile /var/log/maillog.

Don't get me wrong; I don't try to hide some activities, I simply don't want my logfile spammed with meaningless stuff. It's like reducing the log level a bit, which is common practice when choosen wisely.

My comment about "getting rid of the entries" was just a generic call...
I usualy try to avoid using a hammer in order to tighten a screw ;-)

So, if I can't "train" Postfix to just not log this kind of traffic, I probably have to setup some scrip which will copy the maillog to a different location and by using sed I should be able to trim it to my needs.

Is there at all a log level setting for postfix?

Cheers
Juri

Juri:

I wonder, can you modify this probe directly? Change its parameters?
A
Can't miss and the output is clear.
Connection to server_ip 25 port [tcp/smtp] succeeded!
works great and creates no overhead that I could see on the receiving host.

This is a very common problem, and I'd love to see a solution.

In order to properly monitor a service, you have to connect to it and verify that it's actually responding. A simple TCP connection may not be enough. I've seen plenty of cases where a hung service or a crashed subcomponent would not affect the TCP listener, so a simple TCP port test in Nagios/Icinga/OpenNMS/etc. would not detect the service outage.
The logs are sent by Postfix to the syslog service and from there to the log file(s). syslogd can filter on source, facility and severity, but (AFAIK) not on the contents of the entry itself. That leaves Postfix.

I've done a casual search of the Postfix documentation, but couldn't find any setting that would make connections from specific hosts exempt from logging. However, I did find the exact opposite; the debug_peer_list allows you to define one or more hosts as candidates for debug logging, and there's a debug_peer_level setting that controls the log level for those hosts. You may be able to (ab)use this setting by defining the monitoring server as a "debug peer" and set the logging level to 0, if level 0 happens to mean "no logging".

But really, Postfix ought to have a configuration setting that would prevent connections from certain hosts from being logged. Perhaps you could post a feature request to the postfix-devel mailing list? We can't be the only ones that could use such a feature.
That would be a really ugly hack. And it's not really doable, since the logging daemon keeps the log files open at all times. You could conceivably add extra functionality to the logrotate script/process, but that would only affect archived logs ... but no. Seriously, don't even think about it. :)

Every 30 seconds is overdone. Can you change the monitoring interval to 5 minutes?
And perhaps you can configure a short retry interval, say 1 minute?

Sorry for not answering earlier, been quite busy the past few days.

@Habitual
As Ser Olmy pointed out, a simple TCP connect is not enough for us, we really want Postfix to respond to our probes

@Ser Olmy
Thanks for getting into this. I tried you interesting idea of setting a debug_peer_level of 0, unfortunately without luck:
After a postfix reload I get: postfix: fatal: invalid debug_peer_level parameter value 0 < 1
And yes, I figured that cleaning the live log with any sort of hack would not be such a great idea. Apart from being a dirty hack, I would also have to stop postfix before I could even touch the log, which I learned later, and that's definitely a show stoper. Well, I was frustrated and thought, that the hammer tactics might be a suitable solution of a last resort in this special case ;-)

@MadeInGermany
We could argue what is the "right" timing to probe a service, but that doesn't solve the issue, it just reduces the amount of log entries.
Since we also want to catch short outages, we rather leave it at 30 sec with 30 sec retry.
But thanks for adding your view.


What next?
----------
I'll have a look at the syslogd options. Maybe there is something I can use in order to strip the lines in question or maybe pipe the output somewhere else, strip the crap and re-inject it to syslogd and have it then delivered to the postfix maillog. But then again, that's again some sort of hack, which is probably pure overkill just to get rid a few log entries...

If that doesn't help, I'll probably go with a solution similar to the one outlined here: http://stackoverflow.com/questions/1...-write-maillog

I won't touch the live logs, promised ;-)
Rather setting up some cron job to copy the live logs to a working directory every now and then and then have the above linked solution applied in order to strip the logs from the bloat.

If anyone has a better solution, I'm all ears.

Thanks all for you assistance so far, much appreciated.

Cheers
Juri