on Wed, Sep 29, 2004 at 01:16:06PM +0800, Arne G??tje (?????????) (20030910antispam@gmx.net) wrote:
> On Tuesday 28 September 2004 17:17, Karsten M. Self wrote:
> > I've been working with ASN and CIDR data associated with spam
> > received via my ISP account. While the specific findings I've got
> > may be interesting, the methods are of more general use.
> >
> > Short answer: you can classify incoming mail using its IP into its
> > network of origin, with a DNS query.
>
> [snipped a lot of interesting stuff]
>
> may I ask you if you can provide a simple way of looking up the legal
> mailservers of those networks and put them on a whitelist?
There's no surefire way of doing this.
See my followup to Marco: Joe St. Sauver reports on methods, one
solution is to use "hints" offered through name resolution. While
practices aren't standardized, many US ISPs will encode dialup or DSL
lines within the hostname, e.g.: 'adsl' or 'pop...'. Likewise, many
mailservers are identified as "smtp.<domain>" or "mail.<domain>" or
similar.
For the worst-offending netblocks, I'm not sure this is going to help
you. These are simply rogue ISPs with no business being on the Net.
The false-positive rate implied by blocking them is going to be very,
very low, and the benefits of blocking, very high.
And I'll iterate for the nth time: DO NOT use my values. Generate your
own. It's trivial:
- Get a corpus of known spam and known ham.
- Parse headers for the IP feeding your mailserver.
- Query these IPs to get ASN and CIDR.
- Count occurances of ASNs and CIDRs within each corpus. Note any
matches.
I did this yesterday as I was composing my post using simple shell
tools. 'formail' and 'grep' to get headers, awk, sed, or perl to
isolate the IPs (hell, Perl can probably do all of it), and bash calling
a few commands. Not even scripted, just tossed off on the command line.
> I surely only want to block dialup / DSL / Cable accounts of homeusers
> with bot infected Windoze machines, but allow regular users who use
> the companies' legal mailservers.
If the ISP in question is bulk-selling bandwidth directly to spammers,
you'll want to revisit your assumptions.
Of course, on your network, your rules.
My experience, having received, filtered, analyzed, and reported tens of
thousands of spams over the past ten months, is that the organizations
topping my list either have no interest or no capacity to clean their
networks. If they're not going to make that effort, the rest of the
Net is, in my very arrogant opinion, fully at rights to clean up the
traffic for them by dropping it on the floor.
That said, Joe St. Sauver discusses some very limited whitelisting to
allow specific known good nodes to communicate, even from hostile
networks.
Too: for many people and organizations associated with these nets
(Korea, China, Taiwan, Australia, Brazil...), the MO has become to use
services outside their own network for email access, simply because
their own nets are so unreliable.
Third: because you're rejecting at SMPT time, the transaction becomes
deterministic. Rather than mail falling into a black hole (as it
increasingly often does), false-positive blocking is immediately
apparent. The person sending such mail can take steps to establish
contact out-of-band.
> If you would block / filter whole networks without those whitelist,
> you would block more legal users than you want.
While I ask you to believe me when I say that for 4-10 networks, your
false-positive rate would be exceedingly low, I'll encourage you to do
the legwork yourself to determine this. You're arguing rhetorically.
I'm showing data: the volume of spam is immense. The volume of
legitimate traffic, very, very, very low.
> The top asian networks on your list are those providers with the vast
> majority of users here in Asia. I'm very aware of the fact that they
> don't give a shit about spammers. But batting off whole provider
> networks without letting the legal mailservers pass is IMHO not The
> Right Way (TM).
We fundamentally disagree here. I'm a strong believer in incentive
systems. Particularly those which are effective in rewarding the
behavior desired, and discouraging the behavior _not_ desired.
Point-of-origin blacklisting associated with specific organizations
exhibiting a pronounced lack of will / initiative / interest /
capability in dealing with the spam problem has repeatedly proven
effective. At least in getting spam off of _those_ systems. The most
recent success is SAVVIS, only the latest in a long line of conversions.
Additionally, as the spam load increases to the point that
content/context based filtering becomes prohibitive, more organizations
will turn to point-of-origin rules. The incentives for keeping your
network clean will increase.
As for those unfairly blocked, I can only suggest making clear your
concerns to the network management and seeking out alternatives. While
I realize that for raw connectivity, there often are not locally
accesible alternatives, the Internet is desined such that you should be
able to access reliable, respected services offered elsewhere by various
protocols. Be it webmail, SMTP-auth, SSH tunnels, or other means.
That is your problem.
Peace.
--
Karsten M. Self <kmself@ix.netcom.com> http://kmself.home.netcom.com/
What Part of "Gestalt" don't you understand?
The dog returns to it's vomit.
Attachment:
signature.asc
Description: Digital signature