Re: best practices for fighting spam with Debian?

[Daniel Pocock <daniel@pocock.pro>]

On 15/12/16 21:47, Sven Hartge wrote:
> Brian <ad44@cityscape.co.uk> wrote:
>> On Thu 15 Dec 2016 at 10:41:58 +0100, Sven Hartge wrote:
> 
>> [..Snip...]
> 
>>> This seems all very complicated (it is), but because of the environment
>>> I work in (University) it is very important for us (and our users) to
>>> have more control over which email is rejected, trying to minimize false
>>> positives as much as possible while also trying to detect as much SPAM
>>> as possible. A very fine balance, to say the least.
> 
>> It is (as you say) complicated but
> 
>> 1. Why is it important for *your university*?
>> 2. Why is it important for your users?
> 
> Our users (mostly professors) can sometimes be a bit ... particular
> concerning mail reception and possible false positives. 
> 
> This is why we built the whole filtering stuff mostly ourselves instead
> of "outsourcing" it to a ready-made appliance. This would of course been
> a much quicker path but you lose control of the inner workings this way.
> 
> Also the whole internal spam filtering harness has been built over the
> last 15 years and at the beginning such filtering appliances where not
> even a thing.
> 
>> The two are are not the same. For a start, users have no control over
>> your mail server. Secondly, users might want to receive every mail they
>> are sent, irrespective of what *you* think of its spamminess. After all,
>> it is their mail, not yours. There is no "fine balance" as far as a
>> user is concerned. Either they get the mail or they don't.
> 
>> Users can control what mail they receive, of course. Why not leave it
>> up to them to decide?
> 
> This is why the filter is configurable by the user and the user may also
> disable it completely, if s/he so choses. (Few do and many of those who
> chose to disable it reenable it after a short while.)
> 
> There is a fine balance between protecting users from trojans, phishing
> and virus infections on one side and allowing all legitimate mail on the
> other side.
> 

A key point to consider for any filtering is the user experience when a
legitimate message is not delivered.

This includes the following:

- does the sender know their mail was not delivered and do they get a
reasonable explanation?  I've heard there are spam filters who give a
"user doesn't exist" error which is somewhat disrespectful to genuine
senders.  If the messages end up in a junk folder the sender has no
idea.  Personally I think it is better to graylist or reject messages
with a basic explanation like "Your message could not be delivered
because of local policy"

- does the recipient have to wade through a junk mail folder looking for
things that went there by mistake?  In this case, I feel the spam
problem hasn't really been solved at all, the user still has to do the
work sifting through spam to find the ham and it is still a painful
overhead for them.

In some countries and certain industries a lot of companies have simply
started using cloud solutions from some of the large vendors.  They tend
to get all the messages from companies in their industry on the same
platform, but I noticed that one of them was incorrectly putting mail
that had come from one of my Debian mail servers into their junk folder.
 Their IT manager spent a lot of time chasing the vendor to find out why
this was happening and they never got any valid answer.  It was
frustrating for them and for people sending the mail.  The vendor would
just say that they have to keep their policies secret.

Regards,

Daniel