[Sqlgrey-users] Excessive X-Greylist headers

[Alex]

Hi,

It seems when a message is sent to multiple recipients, multiple
X-Greylist headers are added. When there are dozens of these, it
appears some systems are rejecting it due to the header being too
large.

How can I configure it to only include one copy of the X-Greylist header?

I'm using postfix. Is it possible to strip off all but one copy of the header?

X-Greylist: whitelisted by SQLgrey-1.8.0

Thanks,
Alex

------------------------------------------------------------------------------

[Karl O. Pinc]

On Mon, 8 Jun 2015 17:12:22 -0400

Post by Alex
Hi,
It seems when a message is sent to multiple recipients, multiple
X-Greylist headers are added. When there are dozens of these, it
appears some systems are rejecting it due to the header being too
large.
How can I configure it to only include one copy of the X-Greylist header?
I'm using postfix. Is it possible to strip off all but one copy of the header?
X-Greylist: whitelisted by SQLgrey-1.8.0

Is this inbound, outbound, or relayed mail?



Karl <***@meme.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

------------------------------------------------------------------------------

[Alex Regan]

Hi Karl,

Post by Karl O. Pinc

Post by Alex
It seems when a message is sent to multiple recipients, multiple
X-Greylist headers are added. When there are dozens of these, it
appears some systems are rejecting it due to the header being too
large.
How can I configure it to only include one copy of the X-Greylist header?
I'm using postfix. Is it possible to strip off all but one copy of the header?
X-Greylist: whitelisted by SQLgrey-1.8.0

Is this inbound, outbound, or relayed mail?

This is from the headers on the destination system. It is received by
our mail relay, where the header is added, then sent to the destination
system where the user reads the mail.

The mail is apparently sent to dozens of recipients via a mail expander
at the corporate office to recipients on this destination system because
there is no indication in the email that it was sent to dozens of
recipients. Apparently the relay is adding an X-Greylist header for each
of the recipients in the email.

I've included a copy of the email here:

http://pastebin.com/1reiWEM3

Any ideas would be greatly appreciated.
Thanks,
Alex


------------------------------------------------------------------------------

[Karl O. Pinc]

On Tue, 09 Jun 2015 09:51:53 -0400

Post by Alex Regan
Hi Karl,

Post by Karl O. Pinc

Post by Alex
It seems when a message is sent to multiple recipients, multiple
X-Greylist headers are added. When there are dozens of these, it
appears some systems are rejecting it due to the header being too
large.
How can I configure it to only include one copy of the X-Greylist header?
I'm using postfix. Is it possible to strip off all but one copy of the header?
X-Greylist: whitelisted by SQLgrey-1.8.0

Is this inbound, outbound, or relayed mail?

This is from the headers on the destination system. It is received by
our mail relay, where the header is added, then sent to the
destination system where the user reads the mail.

You shouldn't be relaying except from trusted systems,
so your smtpd_relay_restrictions should include
something like "permit_mynetworks", in addition
to your check_policy_service that runs sqlgrey.
Then relayed mail won't go through sqlgrey.
(Use smtpd_recipient_restrictions on older
versions of postfix.)

Whitelisting via sqlgrey is the wong approach.

But it looks to me like you've got something
else wrong since I wouldn't think, without
having really investigated, that any piece of
mail should go through sqlgrey more than once.
So you should only get one header no matter what.


Karl <***@meme.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

------------------------------------------------------------------------------

[Alex Regan]

Hi,

Post by Karl O. Pinc

Post by Alex Regan

Post by Karl O. Pinc

Post by Alex
X-Greylist: whitelisted by SQLgrey-1.8.0

Is this inbound, outbound, or relayed mail?

This is from the headers on the destination system. It is received by
our mail relay, where the header is added, then sent to the
destination system where the user reads the mail.

You shouldn't be relaying except from trusted systems,
so your smtpd_relay_restrictions should include
something like "permit_mynetworks", in addition
to your check_policy_service that runs sqlgrey.
Then relayed mail won't go through sqlgrey.
(Use smtpd_recipient_restrictions on older
versions of postfix.)

The check_policy_service is in smtpd_recipient_restrictions. Is that not
correct?

The smtpd_relay_restrictions is as follows:

smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated,
defer_unauth_destination

This is on postfix-2.10.

Post by Karl O. Pinc
Whitelisting via sqlgrey is the wong approach.

It looks like this was auto-whitelisted, because it has seen this
IP/email combination previously, no?

Post by Karl O. Pinc
But it looks to me like you've got something
else wrong since I wouldn't think, without
having really investigated, that any piece of
mail should go through sqlgrey more than once.
So you should only get one header no matter what.

Yes, I agree. It would be great to figure out how to configure it so
trusted networks weren't greylisted.

Sure appreciate any ideas you might have.
Thanks,
Alex

------------------------------------------------------------------------------

[Karl O. Pinc]

On Tue, 09 Jun 2015 10:49:32 -0400

Post by Alex Regan
Hi,

Post by Karl O. Pinc

Post by Alex Regan

Post by Karl O. Pinc

Post by Alex
X-Greylist: whitelisted by SQLgrey-1.8.0

Is this inbound, outbound, or relayed mail?

This is from the headers on the destination system. It is received
by our mail relay, where the header is added, then sent to the
destination system where the user reads the mail.

You shouldn't be relaying except from trusted systems,
so your smtpd_relay_restrictions should include
something like "permit_mynetworks", in addition
to your check_policy_service that runs sqlgrey.
Then relayed mail won't go through sqlgrey.
(Use smtpd_recipient_restrictions on older
versions of postfix.)

The check_policy_service is in smtpd_recipient_restrictions. Is that
not correct?

Yes, that's right.

Post by Alex Regan
smtpd_relay_restrictions = permit_mynetworks,
permit_sasl_authenticated, defer_unauth_destination

I greylist here too. Keeps more spam out.

Post by Alex Regan
This is on postfix-2.10.

Post by Karl O. Pinc
Whitelisting via sqlgrey is the wong approach.

It looks like this was auto-whitelisted, because it has seen this
IP/email combination previously, no?

Then it must be that the email is exiting postfix and
passing through amavis and being re-injected back
into postfix. You'll need to make sure that amavis
is either on a trusted network (127.0.0.1) or
uses lmtp not smtp or has some other way of getting the mail back
into postfix that does not get greylisted.

I'm not really focusing here so I could be off
on the wrong track. But this is what it looks
like offhand. (And I don't remember anything about amavis
any more.)


Karl <***@meme.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

------------------------------------------------------------------------------

[Alex Regan]

Hi,

Post by Karl O. Pinc

Post by Alex Regan
smtpd_relay_restrictions = permit_mynetworks,
permit_sasl_authenticated, defer_unauth_destination

I greylist here too. Keeps more spam out.

Okay, interesting. I'll try that too.

Post by Karl O. Pinc

Post by Alex Regan
This is on postfix-2.10.

Post by Karl O. Pinc
Whitelisting via sqlgrey is the wong approach.

It looks like this was auto-whitelisted, because it has seen this
IP/email combination previously, no?

Then it must be that the email is exiting postfix and
passing through amavis and being re-injected back
into postfix. You'll need to make sure that amavis
is either on a trusted network (127.0.0.1) or
uses lmtp not smtp or has some other way of getting the mail back
into postfix that does not get greylisted.

All of the networks involved are on the amavis "TRUSTED" or in @mynetworks.

I'm not doubting what you're saying, but here's a thread that seems to
indicate it's the expected behavior:

http://lists.policyd.org/pipermail/users_lists.policyd.org/2007-April/001426.html

Here's another one from Lionel discussing:

http://sourceforge.net/p/sqlgrey/mailman/sqlgrey-users/thread/***@bouton.name/

Could that be the solution here as well?

I'm going to have to use postfix to strip them because I believe it's
causing some mail to be rejected because the headers are too large.

Thanks again,
Alex

------------------------------------------------------------------------------

[Karl O. Pinc]

On Tue, 09 Jun 2015 18:18:29 -0400

Post by Alex Regan

Post by Karl O. Pinc

Post by Alex Regan
smtpd_relay_restrictions = permit_mynetworks,
permit_sasl_authenticated, defer_unauth_destination

I greylist here too. Keeps more spam out.

Okay, interesting. I'll try that too.

On reflection, no, this is not a good idea
in the general case. You need to be careful
with this and if you're not using it now
I'd be leery about adding it.





Karl <***@meme.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

------------------------------------------------------------------------------

[Karl O. Pinc]

On Tue, 09 Jun 2015 18:18:29 -0400
Alex Regan <

Post by Alex Regan

Post by Karl O. Pinc

Post by Alex Regan
This is on postfix-2.10.

Post by Karl O. Pinc
Whitelisting via sqlgrey is the wong approach.

It looks like this was auto-whitelisted, because it has seen this
IP/email combination previously, no?

Then it must be that the email is exiting postfix and
passing through amavis and being re-injected back
into postfix. You'll need to make sure that amavis
is either on a trusted network (127.0.0.1) or
uses lmtp not smtp or has some other way of getting the mail back
into postfix that does not get greylisted.

All of the networks involved are on the amavis "TRUSTED" or in
@mynetworks.
I'm not doubting what you're saying, but here's a thread that seems

Looks like what I'm saying is wrong. The policy
daemon is called for each destination address.
See the postfix SMTPD_POLICY_README.

Post by Alex Regan
Could that be the solution here as well?

"That" being rcpt_awl? Dunno.

Looks like as of postfix 2.1 there is an "instance"
variable passed to the policy daemon that could be used
to keep from adding multiple header lines. It's not
clear how this would work and it's not implemented
in sqlgrey.

Something for somebody's todo list.

Post by Alex Regan
I'm going to have to use postfix to strip them because I believe it's
causing some mail to be rejected because the headers are too large.

That sounds like a sensible approach.

Since the policy check is smtpd you ought to be able
to use header_checks = pcre:/etc/postfix/header_checks
with file content of:

/^X-Greylist: whitelisted by SQLgrey/ IGNORE

I'd bet there's an even more clever way to
"collapse" all the identical entries but
haven't investigated.

Regards,



Karl <***@meme.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

------------------------------------------------------------------------------

[Alex Regan]

Hi Karl,

Post by Karl O. Pinc

Post by Alex Regan
I'm not doubting what you're saying, but here's a thread that seems

Looks like what I'm saying is wrong. The policy
daemon is called for each destination address.
See the postfix SMTPD_POLICY_README.

Post by Alex Regan
Could that be the solution here as well?

"That" being rcpt_awl? Dunno.
Looks like as of postfix 2.1 there is an "instance"
variable passed to the policy daemon that could be used
to keep from adding multiple header lines. It's not
clear how this would work and it's not implemented
in sqlgrey.
Something for somebody's todo list.

Do you think sqlgrey is still the best option for doing greylisting with
postfix, or are is there something else we should be looking it?

I originally thought policyd was an alternative, but it looks to be more
of a front-end or management system that just uses apps like sqlgrey.

Post by Karl O. Pinc

Post by Alex Regan
I'm going to have to use postfix to strip them because I believe it's
causing some mail to be rejected because the headers are too large.

That sounds like a sensible approach.
Since the policy check is smtpd you ought to be able
to use header_checks = pcre:/etc/postfix/header_checks
/^X-Greylist: whitelisted by SQLgrey/ IGNORE
I'd bet there's an even more clever way to
"collapse" all the identical entries but
haven't investigated.

Yes, that's exactly how I'm currently doing it.

Thanks again,
Alex

------------------------------------------------------------------------------

[Karl O. Pinc]

On Wed, 10 Jun 2015 13:18:49 -0400

Post by Alex Regan
Hi Karl,

Post by Karl O. Pinc
Looks like as of postfix 2.1 there is an "instance"
variable passed to the policy daemon that could be used
to keep from adding multiple header lines. It's not
clear how this would work and it's not implemented
in sqlgrey.
Something for somebody's todo list.

Do you think sqlgrey is still the best option for doing greylisting
with postfix, or are is there something else we should be looking it?
I originally thought policyd was an alternative, but it looks to be
more of a front-end or management system that just uses apps like
sqlgrey.

I don't have an informed option, not really knowing what
else is out there. My guess is that if you have something
working it's not really worth the time spent to switch.

I like sqlgrey because it works with Postgres. Not only
do I think Postgres technically superior to MySQL but
MySQL, in my opinion, is not truly Open Source. The
MySQL documentation (and as far as I can tell the
MariaDB documentation) is _not_ open source. A database
is no good without documentation, which makes forking
a db product with no documentation a huge problem.

It took MariaDB _years_ to get decent documentation.
(And near as I can tell they still don't have
a document for each release, making it hard to tell
just how the version you happen to be running
actually works.)

If it can't be forked it's not open source.


Karl <***@meme.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

------------------------------------------------------------------------------