On 2014-09-22 05:29:10, Vincent Lefevre wrote:
> On 2014-08-26 14:28:48 -0700, Antoine Beaupré wrote:
>> Apache, at least in Wheezy, seems to be configured by default to keep 52
>> log files, rotated on a weekly basis, meaning that logs are kept for a
>> year.
>>
>> This is a long time to keep longs. It exposes our users unduly to
>> surveillance and privacy breaches.
>
> Not your users, but people who connect to the web server. But the
> French law requires (required?) / advises to keep the logs for one
> year. There's a discussion in French here:
>
> http://forum.ovh.com/archive/index.php/t-47594.html
>
> Basically this is needed when:
> * Users can create contents.
> * In case of security breach, when someone can do bad things
> via Apache only.
Ouzbekistan law may also require providers to send their logs directly
to the state and install backdoors into their servers, are we going to
do that for all of Debian by default?
It is the provider's responsability to comply with the local laws, not
Debian's, as it is impossible to make a configuration that will work
with every local law.
>> It also means a lot of data to keep on disk for busy webservers. For any
>> moderately to high traffic webserver, this can actually fill up /var
>> pretty fast. For example, a server with an average of 12 hits per
>> second:
>>
>> http://stats.koumbit.net/koumbit.net/ceres.koumbit.net/apache_accesses.html
>>
>> ... accumulates around 30MB *per day*. That means 11GB per year.
>
> Everyone says that disk space is cheap.
I don't. Do you?
> So, this is a very poor argument. Moreover old logs are compressed, so
> that it isn't 11GB per year, but much smaller. With gzip compression
> (which is not very good), I get more than a 10x compression. So, in
> practice, 30 MB per day would mean around 1 GB of disk space on the
> previous default of one year, possibly less.
Those logs were compressed. I know about gzip compression, thank you,
and it is actually pretty decent for log files.
>> I suspect the default partitionning would not allocate enough space
>> for /var at all on most systems to cover for that.
>
> By default, the Debian installer creates a single partition (unless
> this has changed recently).
Ah. I am surprised by that, but I would assume that people would create
a separate /var on server installs. In our experience, we've had
webservers run out of space on /var a few times.
>> I would suggest following the policies set for /var/log/syslog, which
>> are rotate daily and keey 7 days.
>
> Not everyone has such a busy webserver.
Not everyone lives in a country that forces their providers to spy on
their users. Yet anyone can be a victim of massive visits on their
website (aka "slashdotting") which will basically fill up the drives,
regardless of the country they live in.
(Arguably, "slashdottings" themselves are a difficult problem to deal
with and may occur only within a day so log rotation will not help much,
but daily log rotation will certainly be better than weekly to deal with
this.)
We don't want to implement policies that make it difficult to run a
popular webserver - it's the whole point of those things anyways.
> IMHO, the default log rotation should be changed back to 1 year,
> at least to protect users in case of legal matters. Alternatively,
> size-based log rotation could be used, e.g. with:
>
> rotate 15
> size 100M
I think keeping logs does not protect users, it actually exposes them to
undue surveillance. When speaking of "users" here, I refer also to the
visitors of the website, which never agreed to install debian, choose
how much logs are kept and so on. We have a responsability towards those
as well. I prefer to refer to "operators" or "admins" for people that
have the power to do the above configuration.
Operators that want to comply with legal legislation should get a lawyer
and review their storage and compliance policy accordingly. Just keeping
"52 weeks" of logs is hardly legal compliance, even in france you need
to do more than this to comply with the authorities (for example there
are specific delays by which you need to answer).
Telling our operators that they can just install debian by default and
assume legal compliance worldwide is actually hurting them and exposing
them to undue legal complications.
Also, the above configuration, on small sites, could even mean keeping
logs even longer than the original configuration. On big sites, it will
not respect the legal requirements.
I would have proposed to keep no logs at all if I didn't know this would
have created a huge backlash. Furthermore, I think it makes sense to
keep a minimal amount of logs to help people understand what's happening
on a daily basis, so for me 14 days is a compromise. We'll still deploy
a 5 day retention policy at Koumbit, and I know of a lot of groups that
simply censor IPs in the logs altogether, with no ill legal effect or
managmeent problems.
Please keep the change in. It is a good compromise.
A.
--
A man is none the less a slave because he is allowed to choose a new
master once in a term of years.
- Lysander Spooner
Attachment:
pgpa0zz4VD9Y_.pgp
Description: PGP signature