[Sqlgrey-users] auto-whitelisting google/gmail/yahoo

[Alex Regan]

Hi,

It appears the default installation includes an auto-whitelist entry for
google.com and yahoo.com.

Is there a particular reason for this that would prevent me from
removing it? I'm seeing quite a bit of spam that I believe could
otherwise be blocked if the whole domain wasn't whitelisted.

Ideas greatly appreciated.
Thanks,
Alex

[Karl O. Pinc]

On Wed, 01 Jul 2015 20:39:57 -0400

Post by Alex Regan
It appears the default installation includes an auto-whitelist entry
for google.com and yahoo.com.
Is there a particular reason for this that would prevent me from
removing it? I'm seeing quite a bit of spam that I believe could
otherwise be blocked if the whole domain wasn't whitelisted.

The google and yahoo servers are going to obey the rules
and retry after an appropriate interval and deliver
the spam anyway. You may as well reduce the load and
accept it immediately.

They may or may not also resend from a different box,
doing a round-robin on outbound mail gateways.
Mail (probably) eventually gets delivered when all the boxes (IPs)
are whitelisted. But it means that legitimate mail
can get overly delayed. (Dunno who's doing this any more.)

Regards,

Karl <***@meme.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

[Alex]

Hi,

Post by Karl O. Pinc
On Wed, 01 Jul 2015 20:39:57 -0400

Post by Alex Regan
It appears the default installation includes an auto-whitelist entry
for google.com and yahoo.com.
Is there a particular reason for this that would prevent me from
removing it? I'm seeing quite a bit of spam that I believe could
otherwise be blocked if the whole domain wasn't whitelisted.

The google and yahoo servers are going to obey the rules
and retry after an appropriate interval and deliver
the spam anyway. You may as well reduce the load and
accept it immediately.

Ah, right, makes sense.

Post by Karl O. Pinc
They may or may not also resend from a different box,
doing a round-robin on outbound mail gateways.
Mail (probably) eventually gets delivered when all the boxes (IPs)
are whitelisted. But it means that legitimate mail
can get overly delayed. (Dunno who's doing this any more.)

I'm using db_cluster to avoid this. round-robin issue. This was one of
the main reasons I chose sqlgrey over the others like postgrey. Is
this not what you're talking about?

Thanks,
Alex

[d***@hacker.dk]

Hi Alex..

Post by Karl O. Pinc
They may or may not also resend from a different box,
doing a round-robin on outbound mail gateways.
Mail (probably) eventually gets delivered when all the boxes (IPs)
are whitelisted. But it means that legitimate mail
can get overly delayed. (Dunno who's doing this any more.)
I'm using db_cluster to avoid this. round-robin issue. This was one of
the main reasons I chose sqlgrey over the others like postgrey. Is
this not what you're talking about?

No db_cluster (or a central db for that matter) solves the problem of 1
foreign IP hitting multiple servers at YOUR end. Not multiple foreign
servers, with different IP's, hitting your mail server(s).

sqlgrey stores the ip of the sender, either full or as partial, in the
db. But if that ip changes, sqlgrey sees it as an entirely new
"connection" and has to add that one as well. So now sqlgrey has 2 new
connections from the same mail-cluster and both new ip's must resubmit
to be whitelisted and be able to send mail through sqlgrey. Hence; they
are treated as 2 different mail setups.


One always has to remember the purpose of SQLGrey & Greylisting. The
purpose is not filtering spam. The purpose is filtering out mail-senders
that aren't real mailservers. if it IS a real mailserver but it still
sends spam, that is NOT something sqlgrey can do anything about. Thats a
job for other spam-filtering mechanisms.

So, if we KNOW that eg. google.com are actually real mailservers and
that they will alway behave as such, then there is no danger of
whitelisting, as you are only whitelisting the fact that they ARE real
mailservers; not that they arent sending spam.
And as Karl points out, you save time for the clients by whitelisting,
as they dont have to do the whole resubmit-dance to determine that they
are in fact a real mailserver, when we already know it is.

Post by Karl O. Pinc
Thanks,
Alex

Regards
- Dan

[Bruce Bodger]

Good day,

I'm wondering how many of you have implemented the sqlgrey mod described
here... http://www.hyllander.org/sqlgrey_whitelisting It's not very
often that I receive a call from a customer informing me that the email
that he attempted to send to us resulted in an error message (from his
perspective). And the last time was when he was replying to a message
that I sent to him! Of course, the "auto-whitelisting of recipient
addresses" wouldn't help a bit until someone in my domain writes to a
customer but it might add an additional layer of automation. I realize
that the problem relates to broken mail servers but it's our customers
who are inconvenienced and our business that might suffer.

Does anyone have experience with this auto-whitelisting of recipients mod?

Thanks,
~Bruce

[Lionel Bouton]

Hi Bruce,

Post by Bruce Bodger
Good day,
I'm wondering how many of you have implemented the sqlgrey mod described
here... http://www.hyllander.org/sqlgrey_whitelisting It's not very
often that I receive a call from a customer informing me that the email
that he attempted to send to us resulted in an error message (from his
perspective). And the last time was when he was replying to a message
that I sent to him! Of course, the "auto-whitelisting of recipient
addresses" wouldn't help a bit until someone in my domain writes to a
customer but it might add an additional layer of automation. I realize
that the problem relates to broken mail servers but it's our customers
who are inconvenienced and our business that might suffer.

It's unlikely your business will suffer: if your customer reports this
kind of errors, he probably have them with multiple other destinations
(unless his/her admin was temporarily playing with parameters of the
mail server) as greylisting and temporary errors are commonplace.

If you aren't prepared to explain that everything is fine there is
support for whitelisting your customer mail servers (instead of his
domain or addresses as implemented in the patch above which would make
easy to circumvent greylisting by simply using fake addresses derived
from your known customers).
Just add entries to /etc/sqlgrey/clients_fqdn_whitelist.local or
/etc/sqlgrey/clients_ip_whitelist.local depending on how you want to
whitelist (by fqdn or ip). You don't have to restart SQLgrey, these
files are automatically reloaded when modified.

Best regards,

Lionel

[d***@hacker.dk]

Just a side note.

Post by Bruce Bodger
often that I receive a call from a customer informing me that the email
that he attempted to send to us resulted in an error message (from his

If you by "error message" mean that the sender receives a mail that
looks like a bounce (but isn't), I eliminated most of those at my old
job, by using
reject_code = 451

which is actually why i added the reject_code option in the first place :)..

- Dan

[Karl O. Pinc]

On Fri, 3 Jul 2015 11:13:03 -0500

Post by Bruce Bodger
I'm wondering how many of you have implemented the sqlgrey mod
described here... http://www.hyllander.org/sqlgrey_whitelisting

Some thoughts, take them for what you will:

If it were me I'd avoid modifying sqlgrey and write a little
postfix filter that frobs the database whitelist directly.
I'm no advocate of re-inventing the wheel, and honestly
didn't read the article in detail, but simpler seems better
than more complicated.

On a related note, to actually solve the problem you must
know exactly what the problem is. It's not clear from your
description just why there was an issue in the first place.

Karl <***@meme.com>
Free Software: "You don't pay back, you pay forward."
-- Robert A. Heinlein

[Bruce Bodger]

Thanks for your feedback Karl, Dan, and Lionel..

Post by Karl O. Pinc
On Fri, 3 Jul 2015 11:13:03 -0500

Post by Bruce Bodger
I'm wondering how many of you have implemented the sqlgrey mod
described here... http://www.hyllander.org/sqlgrey_whitelisting

If it were me I'd avoid modifying sqlgrey and write a little
postfix filter that frobs the database whitelist directly.
I'm no advocate of re-inventing the wheel, and honestly
didn't read the article in detail, but simpler seems better
than more complicated.

I understand your perspectives. I've been running sqlgrey since 2009.
I was just wondering if you saw this issue as a big deal and, obviously,
you don't. As soon as I hear from a recipient having any problem sending
us email I check the mail log and then add their domain to
clients_fqdn_whitelist.local if I see that they've truly connected.
Again, thanks for the replies. BTW, I've kept the reject_code = 451
default since original installation.

P.S. Even though I've been running sqlgrey for 6 years I've never been
sure what the whitelist requires. Is troubled.com equivalent to
*.troubled.com? My intention is to add all servers and users in the
troubled.com domain. I've been adding both.

Thanks again,
Bruce

[Lionel Bouton]

Post by Bruce Bodger
[...]
P.S. Even though I've been running sqlgrey for 6 years I've never been
sure what the whitelist requires. Is troubled.com equivalent to
*.troubled.com? My intention is to add all servers and users in the
troubled.com domain. I've been adding both.

The whitelisting is about the ips or fqdns of the mail servers
contacting yours and not the addresses of the sender (so they might not
share the same domain). If you are not sure about what they are you
should be able to find them in your postfix logs.

Lionel

[Bruce Bodger]

Post by Lionel Bouton
The whitelisting is about the ips or fqdns of the mail servers
contacting yours and not the addresses of the sender (so they might not
share the same domain). If you are not sure about what they are you
should be able to find them in your postfix logs.

Thank you, Lionel. Understood.

Regards,
~Bruce

[Bruce Bodger]

Post by Lionel Bouton

Post by Bruce Bodger
[...]
P.S. Even though I've been running sqlgrey for 6 years I've never been
sure what the whitelist requires. Is troubled.com equivalent to
*.troubled.com? My intention is to add all servers and users in the
troubled.com domain. I've been adding both.

The whitelisting is about the ips or fqdns of the mail servers
contacting yours and not the addresses of the sender (so they might not
share the same domain). If you are not sure about what they are you
should be able to find them in your postfix logs.

Lionel,

Does this comment from the distributed clients_fqdn_whitelist still apply?

# Note you need the following two lines to allow both
# <lots of mtas>.example.com and example.com
# *.example.com
# example.com

Thank you,
Bruce


------------------------------------------------------------------------------

[Lionel Bouton]

Hi,

Post by Bruce Bodger
[...]
Lionel,
Does this comment from the distributed clients_fqdn_whitelist still apply?
# Note you need the following two lines to allow both
# <lots of mtas>.example.com and example.com
# *.example.com
# example.com

Yes it does.
To be honest I wasn't happy about this little quirk when I coded the
whitelists support and with experience I would not do it again the same
way if I were starting from scratch. IIRC (I didn't look at SQLgrey code
for quite some time) this is needlessly linked to internal SQLgrey
structures involved in the whitelisting: regexp, globs and identity
matching are done separately for performance reasons. I think I should
have used a cleaner syntax and made the effort to translate it to the
internal structures instead of looking for a direct mapping.
As it didn't seem too bad and cleaner solutions would have broken
backward compatibility this was left like this (and the comment was
added to make sure people reading documentation like you were aware of it).

Best regards,

Lionel

------------------------------------------------------------------------------