Re: Let's enable AppArmor by default (why not?)

[intrigeri <intrigeri@debian.org>]

Hi,

a year ago, on August 4 2017, intrigeri wrote:
> tl;dr: I hereby propose we enable AppArmor by default in testing/sid,
> and decide one year later if we want to keep it this way in the
> Buster release.

Here are some data points relevant to this decision making process.
I think we're in a good place to make a decision in November 2018
as planned.

> A proposal
> ==========

> 1. Enable AppArmor by default in testing/sid as soon as feasible in
>    the Buster cycle. […]

AppArmor was enabled by default in November 2017 in our kernel in sid
and the change quickly propagated to testing.

> 2. During a year, watch out for AppArmor related issues and address
>    them in a prompt manner. […]

This happened. See latency stats below.

> 3. A year after AppArmor was enabled by default: evaluate how it went
>    and decide if Buster should be shipped with AppArmor enabled by
>    default or not.

As part of this evaluation process, see the report of the BoF we've
hosted on this topic at DebConf18:
https://people.debian.org/~intrigeri/blog/posts/Report:_AppArmor_BoF_at_DebConf18/

Next step for the AppArmor team: compile the list of remaining issues
we consider as blockers and ensure they're fixed before Buster
is frozen.

>    I commit to do an analysis using BTS data to help make this
>    decision.

AFAICT the BTS currently has very few bugs opened that are caused by
AppArmor and have no patch attached. If the ones you care are not
on our radar, see https://wiki.debian.org/AppArmor/Reportbug#Usertags.

I've analyzed how the AppArmor team has performed once a given bug
appeared on our radar (with the documented usertags) between
2017-08-01 and 2018-08-04, inclusive. tl;dr:

 - 47 bugs were tagged since 2017-08-01 (not counting duplicates);
   I suspect quite a few more issues were not usertagged and are
   therefore not accounted for in my stats; I hope this means the
   maintainer of the affected packages could solve the problem
   themselves somehow;

 - in the vast majority of the cases, the AppArmor team sent a first
   reply on the very day the bug popped up on our radar;

 - in 75% of the cases, a solution was proposed within 8 days once
   the bug was brought to our attention.

If you're into things like standard deviation and want details, see
the statistics section at the bottom of this email :)

reportbug 7.1.8+ reports the enabled LSM so we could compute the
ratio of testing/sid users who opted-out from AppArmor. I doubt
that spending time on it would be the best use of my Debian time
but if you folks feel it's needed to make a decision, I'll do it.

> Here's what popcon says ("Vote" count) for the apparmor binary
> package, that's needed to use AppArmor:

>  * 2015-01:  ~400
>  * 2016-01:  ~700 (+75% in a year)
>  * 2017-01: ~1300 (+85% in a year)
>  * today:    1870 (+44% in 7 months)
     ^^ i.e. 2017-08-04

Update:

   * 2018-08-04 12986 (+594% in a year)
     … presumably because linux-image-* now "Recommends: apparmor".

Statistics
==========

First reply (days)
------------------

 - count:              47
 - variance:           3.22201665124884
 - standard deviation: 1.79499767444107
 - mean:               0.319148936170213
 - per-quartiles:
   * min:                      0
   * 25th percentile:          0
   * 50th percentile (median): 0
   * 75th percentile:          0
   * max:                      12

Fix available (days)
--------------------

That is, either a fix is available and could be applied by the
maintainer, or for very rare corner case, a workaround was provided to
the bug reporter.

 - count:              46
 - variance:           1219.94975845411
 - standard deviation: 34.927779180104
 - mean:               14.695652173913
 - per-quartiles:
   * min:                      0
   * 25th percentile:          0
   * 50th percentile (median): 1
   * 75th percentile:          8
   * max:                      167

Cheers,
-- 
intrigeri