RE: Getting the latest and greatest Linux perf features on every Debian kernel

["Moessbauer, Felix" <felix.moessbauer@siemens.com>]

Hi,

> -----Original Message-----
> From: Ian Rogers <irogers@google.com>
> Sent: Monday, November 22, 2021 3:18 AM
> To: debian-kernel@lists.debian.org; waldi@debian.org
> Cc: linux-perf-users <linux-perf-users@vger.kernel.org>; Arnaldo Carvalho de
> Melo <acme@kernel.org>; ben@decadent.org.uk; Jiri Olsa <jolsa@kernel.org>;
> Namhyung Kim <namhyung@kernel.org>
> Subject: Re: Getting the latest and greatest Linux perf features on every Debian
> kernel
> 
> Hi,
> 
> I'd love to package Linux perf as per Arnaldo's prepared tarball for Debian. If I
> could get help on the Debian side it would be great! The actions that are needed
> are:
> 
> 1) package Arnaldo's tarball for Debian, presumably the build in the sources for
> linux-perf-5.10 will be a good starting point. The package will be linux-perf and
> replace the current metapackage of the same name.

The current packaging in Debian does not include the perf python bindings.
Today these bindings are used by more and more tools (e.g. tuned).
For that, we packaged both a versioned module (linux-perf-<...>) and a python wrapper to load it using 'import perf' (linux-base).
These patches are currently under review at Debian salsa [1].

All this would be way simpler if perf bindings do not have to be versioned anymore.
But please also consider that still a lot of people are using 4.19er kernels (don't know since when versioning is no longer required).

[1] https://salsa.debian.org/kernel-team/linux/-/merge_requests/425

> 
> 2) create an updated linux-base that removes /usr/bin/perf
> 
> 3) upload the updated linux-base and linux-perf packages for maintainer
> approval to
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmentors.
> debian.net%2F&amp;data=04%7C01%7Cfelix.moessbauer%40siemens.com%7C
> 4dbf17b41e4542b4c2de08d9ad5e58c3%7C38ae3bcd95794fd4addab42e1495d5
> 5a%7C1%7C0%7C637731443001828013%7CUnknown%7CTWFpbGZsb3d8eyJWI
> joiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C300
> 0&amp;sdata=80jfnANxTuGYzw0vErMUQOlpdUWFrDNP5n9ktaVTBaQ%3D&amp
> ;reserved=0
> 
> 4) some how get the previous Debian linux-perf-[45].* packages removed

This can easily be achieved using the Conflicts field with a linux-base version that is older than your patched one.

> 
> After this, when Arnaldo sends out a new tarball we can update the linux-perf
> package. libbpf and libtraceevent are currently part of linux/tools, but it would
> be better to depend on the distribution versions, which can be a future update to
> the packaging.
> 
> I'd love some feedback on this plan, whether linux-base/linux-perf have a special
> process, etc. Documentation wise I've seen there is a lot, but:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmentors.
> debian.net%2Fintro-
> maintainers%2F&amp;data=04%7C01%7Cfelix.moessbauer%40siemens.com%7
> C4dbf17b41e4542b4c2de08d9ad5e58c3%7C38ae3bcd95794fd4addab42e1495d
> 55a%7C1%7C0%7C637731443001828013%7CUnknown%7CTWFpbGZsb3d8eyJ
> WIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
> 3000&amp;sdata=Y0L%2Bn%2B0yJqGVnp%2FhCfBVD9V3bZ%2Bffb48vy85jg92M
> oU%3D&amp;reserved=0
> looks fairly to the point.

I really like the plan and I'm also willing to support here.
But please also note that I'm not a Debian sponsor or maintainer of the mentioned packages.

Felix

> 
> Thanks,
> Ian
> 
> 
> 
> On Fri, Nov 12, 2021 at 6:39 AM Arnaldo Carvalho de Melo <acme@kernel.org>
> wrote:
> >
> > Em Thu, Nov 11, 2021 at 10:34:33PM -0800, Ian Rogers escreveu:
> > > Hi,
> >
> > > Debian currently tries to match the Linux perf tool to the version
> > > of the kernel that it is being run upon. Reaching out to Ben
> > > Hutchings, he explained to me that this was done back in 2010 due to
> > > kernel and Linux perf incompatibilities. This was likely the case,
> > > but it was a bug in the Linux perf tool that should have been fixed.
> > > It is the goal of the tool to be backward compatible. A problem with
> > > matching the tool to the kernel version is that users miss out on
> > > new features and fixes (this topic came up in a recent interview of
> > > the maintainer Arnaldo Carvalho de Melo [1]).
> >
> > > Ben Hutchings informs me that making it so that Debian ships the
> > > latest Linux perf tool requires updates both to the linux-base and
> > > linux source packages. The Linux perf tool also has many other often
> > > optional dependencies, like libunwind, libbpf, libpfm4,
> > > libtraceevent, etc. In general, having the dependency will unlock more
> features.
> > > Linux tools has its own copies of libbpf and libtraceevent, and so
> > > these may pose some versioning issues.
> >
> > We can use LIBBPF_DYNAMIC=1 to use the distro libbpf-dev package,
> > which currently is going thru some growing pains as libbpf is 0.x,
> > with several APIs being deprecated, others renamed, and that has been
> > a source of friction, but should be past us with v1.0. Till then the
> > perf codebase is being adjusted to allow it to be seamlessly built
> > with the in-kernel version and with whatever libbpf-devel the distro has.
> >
> > > I think it'd be great to get Debian shipping the latest version of
> > > Linux perf for its users. Hopefully we can agree to change how
> > > Debian packages perf currently and then work out the best way to
> > > package and keep it up-to-date. I look forward to everyone's help and input.
> >
> > I also keep the tarballs available at:
> >
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmirr
> >
> ors.edge.kernel.org%2Fpub%2Flinux%2Fkernel%2Ftools%2Fperf%2F&amp;data=
> >
> 04%7C01%7Cfelix.moessbauer%40siemens.com%7C4dbf17b41e4542b4c2de08d
> 9ad5
> >
> e58c3%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637731443001
> 828013%
> >
> 7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
> I6Ik
> >
> 1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=A2TTg%2BElfRv9bslXw6YmEA1
> R8Iu08Z
> > hFyv6juVhRj24%3D&amp;reserved=0
> >
> > Where there are instructions on how to build this detached tarball.
> >
> > I regularly build perf on lots of distros, including:
> >
> >   debian:9                      : Ok   gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516 ,
> clang version 3.8.1-24 (tags/RELEASE_381/final)
> >   debian:10                     : Ok   gcc (Debian 8.3.0-6) 8.3.0 , clang version 7.0.1-
> 8+deb10u2 (tags/RELEASE_701/final)
> >   debian:11                     : Ok   gcc (Debian 10.2.1-6) 10.2.1 20210110 , Debian
> clang version 11.0.1-2
> >   debian:experimental           : Ok   gcc (Debian 11.2.0-10) 11.2.0 , Debian clang
> version 11.1.0-4
> >   debian:experimental-x-arm64   : Ok   aarch64-linux-gnu-gcc (Debian 11.2.0-9)
> 11.2.0
> >   debian:experimental-x-mips    : Ok   mips-linux-gnu-gcc (Debian 10.2.1-6)
> 10.2.1 20210110
> >   debian:experimental-x-mips64  : Ok   mips64-linux-gnuabi64-gcc (Debian
> 10.2.1-6) 10.2.1 20210110
> >   debian:experimental-x-mipsel  : Ok   mipsel-linux-gnu-gcc (Debian 11.2.0-9)
> 11.2.0
> >
> > Ditto for ubuntu.
> >
> > A complete list can be found in each pull request I send to Linus, see
> > in the last one:
> >
> >
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore
> > .kernel.org%2Fall%2F20211107204947.1394255-1-
> acme%40kernel.org%2F&amp;
> >
> data=04%7C01%7Cfelix.moessbauer%40siemens.com%7C4dbf17b41e4542b4c2
> de08
> >
> d9ad5e58c3%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6377314
> 4300182
> >
> 8013%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzI
> iLCJBT
> >
> iI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=foKAv3e5lm5du1yoLqhzIVf
> vfc7
> > RkZqRAxmfTFkID9U%3D&amp;reserved=0
> >
> > Search for BUILD_TARBALL.
> >
> > There is also the output for 'make -C tools/perf build-test' that
> > tests building with lots of combinations of optional libraries, for
> > instance, with LIBBPF_DYNAMIC=1, etc.
> >
> > - Arnaldo