Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


Need help fighting spam
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Need help fighting spam

daffydaffy Member

Hi folks!

At my company we have a server with spamassassin, clamav and such filtering spam for a couple of servers behind it. It has worked fine for several years, constantly tweaking the filters to prevent false positives and keeping the spam to a minimum. However, for the last couple of months our customers have reported increasing amounts of spam. We have tweaked "everything" but still shit seems to leak through.

We started testing rspamd instead of SA, but there's still a few features missing that won't fit our current implementation of SA towards Postfix.

Do any of you clever people here have any tips for fighting all this?
What information do you need regarding our current setup?

Thanks in advance!

Comments

  • gleertgleert Member, Host Rep
    edited October 2016

    Are you using a control panel? How many domains? Please give us some more info.

  • @jarland OP needs some wisdom on anti-SPAM

    Thanked by 1jar
  • No, we're not using any control panel. The mail goes to the primary mailserver which sends it via the content_filter=amavisfeed:XXX option in Postfix. The filter server uses amavis, SA and Clam to scan the mail, check for blacklists and such. The mail is then sent back to the mailserver if clean and if not, discarded or marked as junk for Dovecot to sort.

    I thought about asking @jarland directly, but wanted to include others as well.

  • jarjar Patron Provider, Top Host, Veteran

    My /var/lib/spamassassin/3.004001/updates_spamassassin_org/10_custom.cf file:

    https://paste.ee/p/s7iE1

    My /etc/mail/spamassassin/local.cf file, just the custom part:

    https://paste.ee/p/q9K4e

  • Thanks, I'll adjust and try them out immediately.

  • You can put greylisting in front of SA if you haven't already done so.

  • Filtering does nothing to stop spam, it just sweeps it under the rug so you don't see it. I do absolutely no filtering and get next to no spam (back when I used to use SpamCop, I'd get 5000/day), because for the last decade I have had a policy for how I share an email address and what happens if it ever ends up in the hands of spammers.

    My suggestion is always that you rethink how you're using email. Stop over-treating the symptoms and start curing the disease. Firewall servers/networks that aren't doing legitimate SMTP connections. Expire email addresses that spammers already have. Stop supporting the pro-spam (i.e., larger numbers + more filtering = better job) approach to "fighting spam".

    Thanked by 1UrDN
  • WebProjectWebProject Host Rep, Veteran

    impossiblystupid said: Filtering does nothing to stop spam, it just sweeps it under the rug so you don't see it. I do absolutely no filtering and get next to no spam (back when I used to use SpamCop, I'd get 5000/day)

    True, but this way you do stop the emails into your email, we do great filter which works for us, simply does 3 point validation and if it's fail it simply decline spam email.

  • UrDNUrDN Member
    edited October 2016

    Use Dspam and never ever subscribe to stealth blocking.

    You may also activate greylisting from time to time.

  • I'm aware that filtering doesn't stop spam in any sense, but when we serve around 14k users one has to asume that 95% of those are morons which leave their email-adresses just about everywhere on the internet. You can't tell all of them to change their email every 6 months because they are stupid, so you have to do something else to help them. My personal email-adress have existed for the better part of two decades and I still only get like 1 spam each month or so. If not less.

  • SPFBL

    https://github.com/leonamp/SPFBL

    english is welcome in the google (support) group

    http://spfbl.net/links.html

  • If you can, try putting it behind a pfSense firewall and add ip blacklists to block known spamming servers.

    Then you can tweak your postfix to have extra protection measures. SPF should be a must for starters but Postfix then add RBL, graylisting etc...

  • OpticalSwooshOpticalSwoosh Member
    edited October 2016

    If possible do some IP checks on the emails. If it's spam and links to CC do send them a abuse report. If nothing happens shoot a PM to @jbiloh he is keen of sorting this out.

    Other network usually just chat shit on twitter till some CTO cough finally resolves it

  • @daffy said:
    We started testing rspamd instead of SA, but there's still a few features missing that won't fit our current implementation of SA towards Postfix.

    What are you looking for that's missing in rspamd?

  • smfsmf Member

    @daffy said:
    Hi folks!

    At my company we have a server with spamassassin, clamav and such filtering spam for a couple of servers behind it. It has worked fine for several years, constantly tweaking the filters to prevent false positives and keeping the spam to a minimum. However, for the last couple of months our customers have reported increasing amounts of spam. We have tweaked "everything" but still shit seems to leak through.

    SpamAssassin uses extensive network checks like DNSBL and URIBL lookups and the best of these (Spamhaus, URIBL and SURBL) are now not free if you do more than 100,000 queries per day and they won't work correctly if your SpamAssassin installation uses DNS servers that forward to ISP nameservers

    You can check if you're blocked by running:

    host -t TXT 2.0.0.127.zen.spamhaus.org

    host -t TXT 2.0.0.127.black.uribl.com

    host -t TXT 2.0.0.127.multi.surbl.org

    All of these lookups should return a test point (127.0.0.x) and a test message - if they don't then you're blocked from querying them.

    The best practise here is to run a local caching nameserver (bind, unbound, pdns_resolver etc.) on each SpamAssassin box or several shared amongst your SA installs.

    We started testing rspamd instead of SA, but there's still a fe features missing that won't fit our current implementation of SA towards Postfix.

    What features is rspamd missing for you? I've been testing it and it does everything that SA does (and more) and is significantly faster than SA due to it's design. I've also contributed a plugin and some rules and I plan to contribute more in the future.

    Do any of you clever people here have any tips for fighting all this?
    What information do you need regarding our current setup?

    My biggest recommendation would be greylisting. Even today, set-up correctly - greylisting can make a huge difference to the amount of spam that leaks through. There are two reasons for this: 1) Even today, some spam cannons do not have a proper retry queue and 2) DNS replication lag is the biggest issue that causes spam to slip though non-greylisting systems. Spamhaus, URIBL etc. will all have lag between detection, listing, replication and DNS cache expiry and spammers today are actively exploiting this to get through existing filters. The greylisting delay is therefore very effective to stop some of this.

    I wrote a paper on greylisting a few years ago that is still accurate today: http://www.fortantispam.com/wp-content/uploads/2013/02/greylisting_whitepaper.pdf

    My 2nd recommendation would be to reject as much at the SMTP stage as possible using DNSBLs and URIBLs which will help with overall system load. Be absolutely 100% sure that you reject any mail from non-existent domains e.g. MAIL FROM:foo@sldkfjsdklfjklwejoewghrjiowf.com should be rejected outright.

    Hope that helps.

    Cheers,

    Steve.

  • @daffy said:
    I'm aware that filtering doesn't stop spam in any sense, but when we serve around 14k users one has to asume that 95% of those are morons which leave their email-adresses just about everywhere on the internet.

    Maybe true, but what other guidance have you given them to keep them from making such mistakes? Do they even have the opportunity to associate multiple email addresses to one user account? Does anything in your filtering process also bring attention to which addresses are getting compromised and increasingly shared?

    You can't tell all of them to change their email every 6 months because they are stupid, so you have to do something else to help them.

    And yet, were their password compromised, they would be expected to change it. Hell, they may even have to change it every 3 months based on nothing but corporate policy. There are obvious infrastructure costs and security implications associated with leaking any information, including email addresses. We'd all be more helpful if we didn't allow people to be stupid about such things.

    My personal email-adress have existed for the better part of two decades and I still only get like 1 spam each month or so. If not less.

    Do you think that's typical, or have you considered you might just be one of the lucky ones? Have you done any sort of experiment on email gathering to get an idea of how spammers operate? I get that doing things different isn't always the easiest thing (e.g., very few email clients make disposable addresses convenient), but the security gains I've seen are still worth it.

  • @daffy said:
    Hi folks!

    At my company we have a server with spamassassin, clamav and such filtering spam for a couple of servers behind it. It has worked fine for several years, constantly tweaking the filters to prevent false positives and keeping the spam to a minimum. However, for the last couple of months our customers have reported increasing amounts of spam. We have tweaked "everything" but still shit seems to leak through.

    We started testing rspamd instead of SA, but there's still a few features missing that won't fit our current implementation of SA towards Postfix.

    Do any of you clever people here have any tips for fighting all this?
    What information do you need regarding our current setup?

    Thanks in advance!

    Hi. I'm developer of a new anti-spam system called SPFBL. This is an open-source project to share reputation in a P2P network with many providers here in Brazil:

    https://github.com/leonamp/SPFBL

    We have a DNSBL service that uses this reputation table:

    dnsbl.spfbl.net

    We have a MX filter service too. I can give you 30 days free for you test service:

    http://www.spfbl.net/dnsbl/english/servicomx.html

    You just need change MX of your domain to our MX if you want test it. But we don't use Spamassassin because too much false positive. We just use Exim+SPFBL+Clamav with 85% of volume filtered.

    Leandro
    SPFBL.net

  • rpcoperpcope Member
    edited October 2016

    @UrDN said:
    Use Dspam and never ever subscribe to stealth blocking.

    You may also activate greylisting from time to time.

    @daffy

    Dspam is really effective and lightweight when it works, but has been pretty painful and finicky in the past. It has had a tendency to segfault occasionally, and using the hash store, the whole thing got clogged up sometimes (especially if you try to train it on a large dataset), eventually became more headache than it's worth. I also remember thinking that it's not actively maintained.

    On the flipside, greylisting has helped a lot for me, when coupled with an instance of spamassassin that's kept up to date. Postgrey is pretty good if postfix is your MTA.

    Do you use DKIM and SPF milters for filtering incoming mail?

    Thanked by 1FrankZ
  • @rpcope said:

    @UrDN said:
    Use Dspam and never ever subscribe to stealth blocking.

    You may also activate greylisting from time to time.

    @daffy

    Dspam is really effective and lightweight when it works, but has been pretty painful and finicky in the past. It has had a tendency to segfault occasionally, and using the hash store, the whole thing got clogged up sometimes (especially if you try to train it on a large dataset), eventually became more headache than it's worth. I also remember thinking that it's not actively maintained.

    On the flipside, greylisting has helped a lot for me, when coupled with an instance of spamassassin that's kept up to date. Postgrey is pretty good if postfix is your MTA.

    Do you use DKIM and SPF milters for filtering incoming mail?

    Yes, we use DKIM and SPF-milters, and that seems to help quite a lot. I also started implementing greylisting, but that blocks quite a few autogenerated invoices from ie. powercompanys, phonecompanys etc.

    I too noticed that DSPAM is no longer actively maintained.

    @smf said:

    @daffy said:
    Hi folks!

    At my company we have a server with spamassassin, clamav and such filtering spam for a couple of servers behind it. It has worked fine for several years, constantly tweaking the filters to prevent false positives and keeping the spam to a minimum. However, for the last couple of months our customers have reported increasing amounts of spam. We have tweaked "everything" but still shit seems to leak through.

    SpamAssassin uses extensive network checks like DNSBL and URIBL lookups and the best of these (Spamhaus, URIBL and SURBL) are now not free if you do more than 100,000 queries per day and they won't work correctly if your SpamAssassin installation uses DNS servers that forward to ISP nameservers

    You can check if you're blocked by running:

    host -t TXT 2.0.0.127.zen.spamhaus.org

    host -t TXT 2.0.0.127.black.uribl.com

    host -t TXT 2.0.0.127.multi.surbl.org

    All of these lookups should return a test point (127.0.0.x) and a test message - if they don't then you're blocked from querying them.

    The best practise here is to run a local caching nameserver (bind, unbound, pdns_resolver etc.) on each SpamAssassin box or several shared amongst your SA installs.

    We started testing rspamd instead of SA, but there's still a fe features missing that won't fit our current implementation of SA towards Postfix.

    What features is rspamd missing for you? I've been testing it and it does everything that SA does (and more) and is significantly faster than SA due to it's design. I've also contributed a plugin and some rules and I plan to contribute more in the future.

    Do any of you clever people here have any tips for fighting all this?
    What information do you need regarding our current setup?

    My biggest recommendation would be greylisting. Even today, set-up correctly - greylisting can make a huge difference to the amount of spam that leaks through. There are two reasons for this: 1) Even today, some spam cannons do not have a proper retry queue and 2) DNS replication lag is the biggest issue that causes spam to slip though non-greylisting systems. Spamhaus, URIBL etc. will all have lag between detection, listing, replication and DNS cache expiry and spammers today are actively exploiting this to get through existing filters. The greylisting delay is therefore very effective to stop some of this.

    I wrote a paper on greylisting a few years ago that is still accurate today: http://www.fortantispam.com/wp-content/uploads/2013/02/greylisting_whitepaper.pdf

    My 2nd recommendation would be to reject as much at the SMTP stage as possible using DNSBLs and URIBLs which will help with overall system load. Be absolutely 100% sure that you reject any mail from non-existent domains e.g. MAIL FROM:foo@sldkfjsdklfjklwejoewghrjiowf.com should be rejected outright.

    Hope that helps.

    Cheers,

    Steve.

    I'll read through your paper and see if I can find something there, thanks!
    Regarding rspamd, I guess it isn't about missing features, more that they seem unable to add headers to outgoing mail. Our filtering adds headers to both spam and non-spam so that the recieving MTA can sort the email accordingly and also blocks outgoing email. I still haven't gotten rspamd to do so. The same goes for discarding spam. I don't want to reject and cause backscatter which somehow seems to be the default for rspamd, and discard just.... doesn't.

  • nerfnerf Member
    edited October 2016

    @daffy said:
    Regarding rspamd, I guess it isn't about missing features, more that they seem unable to add headers to outgoing mail. Our filtering adds headers to both spam and non-spam so that the recieving MTA can sort the email accordingly and also blocks outgoing email. I still haven't gotten rspamd to do so. The same goes for discarding spam. I don't want to reject and cause backscatter which somehow seems to be the default for rspamd, and discard just.... doesn't.

    Ultimately, rejecting/not rejecting mail and adding headers is the job of your MTA- since you mentioned Postfix I suppose you're using Rmilter- in which case if you want to unconditionally add a spam report header you could add extended_spam_headers = true in the spamd section. To disable reject set spamd_never_reject = true.

    If you need headers in some special format you could generate these in rspamd and return them using task:set_rmilter_reply().

    Rspamd is intended to be used on the MX (where rejecting is fine).

    Try #rspamd on Freenode or the mailing list for help.

  • jarjar Patron Provider, Top Host, Veteran

    Reject is good and proper. Bounce is what you want to avoid. I do believe postfix references those in the way that I am.

  • RazzaRazza Member
    edited October 2016

    jarland said: My /etc/mail/spamassassin/local.cf file, just the custom part:
    https://paste.ee/p/q9K4e

    You got a syntax error, to whitelist a domain using whitelist_from you need to write it like whitelist_from *@domain.com, also using whitelist_from could by mistake make SpamAssassin more ineffective.

    I will give you a example lets say a junk email gets send using spoof sender which matches one of the white listed as long as the email gets passed other checks such as rbl lookups configured on the mail server, SpamAssassin will give it a -100 USER_IN_WHITELIST From: address is in the user's white-list which is unlikely other rules such as SPF fail or rules about the email content will be able to offset the -100 .

    I done a mirror edit edit to my version based on your version i changed some domain to use whitelist_from_rcvd which also checks the rDNS of the last received header, its work fine for domain that you know what the most likely valid domain in the rdns would be e.g Gmail outgoing mail get sent via ip with the Rdns set to a value in this format mail-LL0-L000.google.com.

    The domain i changed to whitelist_from_rcvd , i checked the received headers in my mail server logs for email's sent from that domain , to find the most suitable domain to check for in the Rdns.

    Fixed Version https://paste.ee/p/3TPD8

    My Version with whitelist_from_rcvd https://paste.ee/p/uK16z

    Thanked by 1jar
  • mailcheapmailcheap Member, Host Rep
    edited October 2016

    @daffy said:
    Hi folks!

    At my company we have a server with spamassassin, clamav and such filtering spam for a couple of servers behind it. It has worked fine for several years, constantly tweaking the filters to prevent false positives and keeping the spam to a minimum. However, for the last couple of months our customers have reported increasing amounts of spam. We have tweaked "everything" but still shit seems to leak through.

    We started testing rspamd instead of SA, but there's still a few features missing that won't fit our current implementation of SA towards Postfix.

    Do any of you clever people here have any tips for fighting all this?
    What information do you need regarding our current setup?

    Thanks in advance!

    First secure the basics with Postfix checks (helo, sender & recipient restrictions, SPF checks, etc.) and just Spamhaus RBL.

    I recommend an aggressive SA score (4.0) with manual learning for domains/users. It might be a headache during the first week or two but it will stop targeted spam or hard to filter ones in foreign languages while keeping false positives to a minimum as time goes on.

    Pavin Joseph.

  • jarjar Patron Provider, Top Host, Veteran

    Razza said: You got a syntax error, to whitelist a domain using whitelist_from you need to write it like whitelist_from *@domain.com, also using whitelist_from could by mistake make SpamAssassin more ineffective.

    Thanks for correcting me on that. I do always struggle with whitelist, and I try to keep an eye out for spoofs. Mostly this is kind of a fluid section where I try to offset a little bit of the stuff below to keep the valid emails from spam. Sort of a "let's target everything and walk backwards on the false positives" thing. It's a bit of a delicate balance.

    Thanked by 1Razza
  • RazzaRazza Member
    edited October 2016

    jarland said: I try to keep an eye out for spoofs

    I won't worrie too much about that, most hosts sending spam using spoof email will probably listed on one of the major rbl anyway.

    whitelist_from_rcvd works quite well for some domain, Gmail is fine as all the sending IP ptr end in google.com as far as i can tell.

    I would say using whitelist_from_rcvd on Amazon is fine as well as all the email such as Amazon order email, Amazon Web Services,etc come from a IP with the ptr ending in amazonses.com.

    I think most of the Hotmail/live/Outlook domain are fine too as the sender IP ptr end in hotmail.com.

    I think in your case for Mxroute its probably safer just using whitelist_from, just incase legitimate mail dose not get whitelisted.

    Thanked by 1jar
Sign In or Register to comment.