[Sqlgrey-users] sqlstats need explanation

[Len Conrad]

/usr/local/bin/sqlgrey-stats.sh

GREY NEW: 70491
GREY EARLY RECON: 790
GREY RECON OK: 954
GREY DOMAWL: 7805
GREY FROM AWL: 795
WHITELIST: 191
SPAM: 33515
SMTPD GREYLISTED: 4015

... for about 11 hours Monday morning.

I find the smtpd "greylisted for 5 minutes" rejects to be extremely low compared to what I see with postgrey.

For 70K "new/never-seen" triplet, why aren't there 70K smtpd rejects?

Len

[Lionel Bouton]

Post by Len Conrad
/usr/local/bin/sqlgrey-stats.sh
GREY NEW: 70491
GREY EARLY RECON: 790
GREY RECON OK: 954
GREY DOMAWL: 7805
GREY FROM AWL: 795
WHITELIST: 191
SPAM: 33515
SMTPD GREYLISTED: 4015
... for about 11 hours Monday morning.

I don't know sqlgrey-stats.sh so I'm not sure exactly what it should
report. That said the categories seem to match SQLgrey's own log
categories so I assume it makes sums of log lines matching these.

Post by Len Conrad
I find the smtpd "greylisted for 5 minutes" rejects to be extremely low compared to what I see with postgrey.
For 70K "new/never-seen" triplet, why aren't there 70K smtpd rejects?

Why do you think there aren't ? Nothing above supports this claim: for
each "grey new" and "grey early recon" line the server should return a
temporary reject. If it doesn't, then it's a SMTP server configuration
issue.

Best regards,

Lionel

[Len Conrad]

Post by Lionel Bouton

Post by Len Conrad
/usr/local/bin/sqlgrey-stats.sh
GREY NEW: 70491
GREY EARLY RECON: 790
GREY RECON OK: 954
GREY DOMAWL: 7805
GREY FROM AWL: 795
WHITELIST: 191
SPAM: 33515
SMTPD GREYLISTED: 4015
... for about 11 hours Monday morning.

I don't know sqlgrey-stats.sh so I'm not sure exactly what it should
report. That said the categories seem to match SQLgrey's own log
categories so I assume it makes sums of log lines matching these.

Post by Len Conrad
I find the smtpd "greylisted for 5 minutes" rejects to be extremely low compared to what I see with postgrey.
For 70K "new/never-seen" triplet, why aren't there 70K smtpd rejects?

Why do you think there aren't ? Nothing above supports this claim: for
each "grey new" and "grey early recon" line the server should return a
temporary reject. If it doesn't, then it's a SMTP server configuration
issue.

NEW=`egrep -ic "sqlgrey: grey: new:" /var/log/mx1.hctc.net/maillog`
EARLY=`egrep -ic "sqlgrey: grey: early reconnect:" /var/log/mx1.hctc.net/maillog`
RECON=`egrep -ic "sqlgrey: grey: reconnect ok:" /var/log/mx1.hctc.net/maillog`
DOMAWL=`egrep -ic "sqlgrey: grey: domain awl match" /var/log/mx1.hctc.net/maillog`
FRMAWL=`egrep -ic "sqlgrey: grey: from awl:" /var/log/mx1.hctc.net/maillog`
WHITE=`egrep -ic "sqlgrey: whitelist:" /var/log/mx1.hctc.net/maillog`
SPAM=`egrep -i "sqlgrey: spam:" /var/log/mx1.hctc.net/maillog |awk '{print $7}'|sort -n|uniq -i|wc -l`
GLIST=`egrep -ic "Greylisted for 5 minutes" /var/log/mx1.hctc.net/maillog`


Len

[Lionel Bouton]

Post by Len Conrad

Post by Lionel Bouton

Post by Len Conrad
/usr/local/bin/sqlgrey-stats.sh
GREY NEW: 70491
GREY EARLY RECON: 790
GREY RECON OK: 954
GREY DOMAWL: 7805
GREY FROM AWL: 795
WHITELIST: 191
SPAM: 33515
SMTPD GREYLISTED: 4015
... for about 11 hours Monday morning.

I don't know sqlgrey-stats.sh so I'm not sure exactly what it should
report. That said the categories seem to match SQLgrey's own log
categories so I assume it makes sums of log lines matching these.

Post by Len Conrad
I find the smtpd "greylisted for 5 minutes" rejects to be extremely low compared to what I see with postgrey.
For 70K "new/never-seen" triplet, why aren't there 70K smtpd rejects?

Why do you think there aren't ? Nothing above supports this claim: for
each "grey new" and "grey early recon" line the server should return a
temporary reject. If it doesn't, then it's a SMTP server configuration
issue.

NEW=`egrep -ic "sqlgrey: grey: new:" /var/log/mx1.hctc.net/maillog`
EARLY=`egrep -ic "sqlgrey: grey: early reconnect:" /var/log/mx1.hctc.net/maillog`
RECON=`egrep -ic "sqlgrey: grey: reconnect ok:" /var/log/mx1.hctc.net/maillog`
DOMAWL=`egrep -ic "sqlgrey: grey: domain awl match" /var/log/mx1.hctc.net/maillog`
FRMAWL=`egrep -ic "sqlgrey: grey: from awl:" /var/log/mx1.hctc.net/maillog`
WHITE=`egrep -ic "sqlgrey: whitelist:" /var/log/mx1.hctc.net/maillog`
SPAM=`egrep -i "sqlgrey: spam:" /var/log/mx1.hctc.net/maillog |awk '{print $7}'|sort -n|uniq -i|wc -l`
GLIST=`egrep -ic "Greylisted for 5 minutes" /var/log/mx1.hctc.net/maillog`

GLIST counts the rejects where the SMTP server based it's decision on
SQLgrey's result.
It doesn't count mails rejected because they have been rejected by both
SQLgrey and another rule in relevant smtpd_*_restriction configuration
entries that takes precedence (probably because it does a permanent
reject instead of the temporary one SQLgrey tells Postfix to return).
This is expected behavior if you use RBLs (especially if they cover
ranges of residential ip addresses).

If you look into your logs you should see that the messages triggering
the "grey new" logs are permanently refused a short time after in
Postfix logs.

Lionel

[Roddie Hasan]

Post by Len Conrad
NEW=`egrep -ic "sqlgrey: grey: new:" /var/log/mx1.hctc.net/maillog`
EARLY=`egrep -ic "sqlgrey: grey: early reconnect:" /var/log/mx1.hctc.net/maillog`
RECON=`egrep -ic "sqlgrey: grey: reconnect ok:" /var/log/mx1.hctc.net/maillog`
DOMAWL=`egrep -ic "sqlgrey: grey: domain awl match" /var/log/mx1.hctc.net/maillog`
FRMAWL=`egrep -ic "sqlgrey: grey: from awl:" /var/log/mx1.hctc.net/maillog`
WHITE=`egrep -ic "sqlgrey: whitelist:" /var/log/mx1.hctc.net/maillog`
SPAM=`egrep -i "sqlgrey: spam:" /var/log/mx1.hctc.net/maillog |awk '{print $7}'|sort -n|uniq -i|wc -l`
GLIST=`egrep -ic "Greylisted for 5 minutes" /var/log/mx1.hctc.net/maillog`

krweb:/root# cat sqlgrey-stats.sh

NEW=`egrep -ic "sqlgrey: grey: new:" /var/log/maillog`
EARLY=`egrep -ic "sqlgrey: grey: early reconnect:" /var/log/maillog`
RECON=`egrep -ic "sqlgrey: grey: reconnect ok:" /var/log/maillog`
DOMAWL=`egrep -ic "sqlgrey: grey: domain awl match" /var/log/maillog`
FRMAWL=`egrep -ic "sqlgrey: grey: from awl:" /var/log/maillog`
WHITE=`egrep -ic "sqlgrey: whitelist:" /var/log/maillog`
SPAM=`egrep -i "sqlgrey: spam:" /var/log/maillog |awk '{print $7}'|sort -n|uniq -i|wc -l`
GLIST=`egrep -ic "Greylisted for 5 minutes" /var/log/maillog`
echo "GREY NEW:" $NEW
echo "GREY EARLY RECON:" $EARLY
echo "GREY RECON OK:" $RECON
echo "GREY DOMAWL:" $DOMAWL
echo "GREY FROM AWL:" $FRMAWL
echo "WHITELIST:" $WHITE
echo "SPAM:" $SPAM
echo "SMTPD GREYLISTED:" $GLIST

Roddie

[Len Conrad]

---------- Original Message ----------------------------------
From: Lionel Bouton <lionel-***@bouton.name>
Date: Mon, 01 Jun 2009 19:38:38 +0200

Post by Lionel Bouton

Post by Len Conrad
/usr/local/bin/sqlgrey-stats.sh
GREY NEW: 70491
GREY EARLY RECON: 790
GREY RECON OK: 954
GREY DOMAWL: 7805
GREY FROM AWL: 795
WHITELIST: 191
SPAM: 33515
SMTPD GREYLISTED: 4015
... for about 11 hours Monday morning.

I don't know sqlgrey-stats.sh so I'm not sure exactly what it should
report. That said the categories seem to match SQLgrey's own log
categories so I assume it makes sums of log lines matching these.

Post by Len Conrad
I find the smtpd "greylisted for 5 minutes" rejects to be extremely low compared to what I see with postgrey.
For 70K "new/never-seen" triplet, why aren't there 70K smtpd rejects?

Why do you think there aren't ? Nothing above supports this claim: for
each "grey new" and "grey early recon" line the server should return a
temporary reject. If it doesn't, then it's a SMTP server configuration
issue.

ok, found my misconfig:

changed these from "delay" to:

reject_first_attempt = immed
reject_early_reconnect = immed


"greylisted" smtpd log lines are streaming up the screen now.

thanks
Len