On Fri, Nov 01, 2019 at 04:35:24PM -0400, Gene Heskett wrote:
> On Friday 01 November 2019 14:44:07 Gene Heskett wrote:
>
> > On Friday 01 November 2019 13:43:04 tomas@tuxteam.de wrote:
> > > On Fri, Nov 01, 2019 at 01:12:46PM -0400, Gene Heskett wrote:
> > > > On Friday 01 November 2019 12:42:21 tomas@tuxteam.de wrote:
> > >
> > > [...]
> > >
> > > > > https://en.wikipedia.org/wiki/Robots.txt
> > > >
> > > > Did that, blanket Disallow for all didn't stop them. But they are
> > > > spacing out the requests now, so the average traffic is very low.
> > > > I can tolerate that.
> > >
> > > You can block by user agent, that's more drastic.
> > >
> > > If I've been paying attention, you are on apache. Then this might
> > > be relevant:
> > >
> > >
> > > https://httpd.apache.org/docs/2.4/rewrite/access.html#blocking-of-ro
> > >bo ts
> >
> > Unfortunately its starts with a very fuzzy explanation of where to
> > put all those examples. I suspect thats because I probably don't have
> > near all of apache2 installed.
Those are directives for the apache configuration, somewhere under /etc/apache2,
I guess (it's a while since I did Apache -- these days I prefer something
smaller, like lighttpd).
> Nother question, perhaps back on thread. I have 30 or so copies of a
> rotots.txt that should block the bots, but in half an hour after I've
> done an apache2 restart, they are back again.
>
> Does apache2 pay any attention to hosts.deny, and does hosts.deny honor
> CIDR addresses which would allow me to block the whole /24 they are
> coming from. What they are doing is, since they're using all my upload
> bandwidth, qualifies as a DDOS.
That will depend on whether apache is compiled with tcpwrappers (that's
the library implementing the hosts.{allow,deny} policies). I don't
know whether Debian's distribution does that (perhaps others will).
> And at this point I don't care if I play dirty with stuff outside of
> apache2's control.
>
> Ack the man page CIDR working is yes. But there is no clue what log file
> to look at to see if its working to control what apache2 does. That
> would be most helpfull.
>
> And a new one just showed up, a Mac known as bytespider@bytedance.com,
> and he/she got put in hosts.deny. tsk tsk.
IMO Apache config is the cleaner option here, but hey, it's your box :)
For apache, you'll find the log files in /var/log/apache2, I think.
For tcpwrappers... dunno.
Cheers
-- t
Attachment:
signature.asc
Description: Digital signature