[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Packaging crowdsec and some other packages



Hi,

El boulangero <elboulangero@gmail.com> (2020-12-16):
> just answering a few points quickly.

Thanks! :)

(Dropping Alexandre Viau and Shengjing Zhu for the general approach to
other packages.)

> If I understand correctly, you want to focus on crowdsec the program first.
> In this case, starting with 'dh-make-golang make -type program' is the
> right way to go. The source package will be named 'crowdsec', and produce a
> binary package named 'crowdsec'. Later on when you want to provide a
> library package, you will just add a new binary package to debian/control,
> named 'golang-github-crowdsecurity-crowdsec-dev', then add the file
> `debian/golang-github-crowdsecurity-crowdsec-dev.install`, and that's
> pretty much done already.
> 
> Many packages do that, for example nomad, consul, containerd.

Alright, I wasn't too worried about how to add a binary after the fact
(those aren't my first Debian packages), I was just wondering whether
having opted for a program-only approach at first could lead to some
kind of difficulties on the long run.

It seems that shouldn't be the case. :)

> I also think you're right here. All the build depends, ie. packages
> you don't necessarily care about, and that might also be used by other
> programs, are better maintained within the team. So that there's no
> friction to work on it, and everyone from the team can fix or update
> them when needed. All these small packages are "common ground" more or
> less.

Yes, that looks like an ideal situation, was just meaning to check we
shared this view.

> As for the main package crowdsec, that you care about, I guess it also
> makes sense to put yourself as the maintainer, or to create a
> "Corwdsec Packaging Team" as you suggest. Right now I maintain the
> docker.io package, it's the same situation, it lives in the "Docker
> Packaging Team".

Alright. I'm not sure whether we want to be having a specific team for
it, or if it would make most sense to have the repository on Salsa
alongside all other golang-* packages (with the Maintainer not being set
to “Debian Go Packaging Team”). It might be easier to have all packaging
related repositories in the same place, instead of having to switch to a
different team, or some branch of the upstream (GitHub-based)
repository. We'll brainstorm a little more about this.

> > Does everything make sense so far?
> 
> Yep.

Thanks for the review and comments so far, much appreciated!

> > One which seemed easy enough was the following, and the rev-deps:
> >  - golang-github-oschwald-maxminddb-golang
> >  - golang-github-oschwald-geoip2-golang
> >  - syncthing
> 
> So basically, when you update libraries, you should check that the rev
> build deps don't break. We have a tool for that, it's ratt (rebuild all the
> things). It automates rebuilding all the reverse build deps with sbuild,
> while injecting in the build environment the new package.

Oh, great, I was hoping such a thing would exist! May I suggest adding
it somewhere on <https://go-team.pages.debian.net/>? I think I would
have expected to see it mentioned on the packaging.html page, or maybe
on some other “best practices/updating packages” page. Now that I know
it exists and how it's named, I see it's referenced on the ci.html page,
but I hadn't thought about checking that kind of page before, since it
looked like a different topic.

> If ratt succeeds, then you can upload. If you're still unsure, better
> ask on the Debian Go mailing list before uploading, it won't hurt.
> Some packages are sensitive, or have many many reverse build deps.
> Better ask before upload until you're sure :)

I think I'm decently good at the “do when you know, ask when you're not
sure” game (that was one of key aspects I would check when accompanying
people through the NM process); as you have seen, I've already sent a
small heads-up to two maintainers for packages that I'd like to update
in the near future. ;)

I'll come back to those topics once I've practiced ratt a little more!

> If ratt fails (ie. some packages fail to build with the new library
> that you just packaged), then you should investigate why, and fix the
> reverse build-deps for the new package that you intend to upload.

Yeah, I was definitely looking forward to investivating where relevant
before doing anything (checking the golang-github-gin-gonic-gin rev-deps
is one big item on my short-term todo list), I was most interested in
getting to know the relevant tooling. :)

> If ratt fails and you don't care, and upload your package all the
> same, well, nothing will stop you, but it's not nice at all. You're
> putting more work on other people's shoulders. And it can be a lot of
> work. Please don't do that :)

I've been on the receiving end of such things a few years back, when
maintainers would trigger library transitions without any kind of
coordination, leaving other maintainers to pick up and fix the broken
pieces (and release team members to try and draft plans to help them,
also having to deal with entangled transitions, etc.). I think it's fair
to say this isn't my style. ;)

> Apart from that, a few comments from my experience.

Thanks so much for that as well!

> Sometimes, a little patching is much better than a lot of blind
> packaging. For example, if you can patch a feature away, and save
> yourself packaging and maintaining a tree of 10+ dependencies, then
> maybe you should do that. You can see that at work in docker-registry,
> where support for relic and azure was removed. Nobody complained.

I'm already doing that with crowdsec upstream, where packages that
aren't in Debian have been evaluated. That's why we've come to the
conclusion we'd like to upgrade some existing packages, which should
benefit everyone.

I haven't built a graph just yet, but from my manual notes it appears
the dependency tree is quite flat: a few packages have 1 or 2
dependencies themselves, but those “level-2” dependencies are actually
shared between some packages, so it seems easiest to just go the extra
mile and add 1 or 2 packages to the whole lot, instead of trying to
patch things out. :)

> You should exclude the 'vendor' directory if there's one. It's done in the
> Files-Excluded field of debian/copyright. However, for the "big packages",
> it's fairly common to keep a few of the vendored libraries, either because
> it saves us from a complicated build failure, or because some dependencies
> are not packaged. You can see that at work in the debian/copyright file of
> "big packages", once again: nomad, consul, containerd. docker.io, ...
> However, 1) the Files-Excluded list is a bit of a pain to maintain, 2)
> You're out of luck with crowdsec, they don't have a vendor directory.

I think 2 dependencies have such a directory, and I've definitely
spotted that as being something important to keep an eye on, along with
the various licenses, etc. I've seen dh-make-golang detects it and takes
it into account when generating the stub packaging, which I liked a lot!
:)

> Other than that, good luck with crowdsec, and welcome to the Go Team,
> we have many packages and need more hands!

Thanks :)


I should be polishing the initial, assisted packaging for many packages
over the next few days; is it OK for me to create as many repositories
as required while doing so, filing ITPs, and possibly sponsoring myself
(the not-so-usual kibi@debian.org vs. cyril@debamax.com schizophrenia…)
when I have no specific doubts about a given package, or would the team
prefer my going through some kind of internal review before throwing
packages at NEW?

(Again, the packages I'd like to upgrade are going to be subject to more
testing on my side, and coordination with maintainers/previous uploaders
before any action anyway.)


Cheers,
Cyril.
-- 
Cyril Brulebois -- Debian Consultant @ DEBAMAX -- https://debamax.com/

Attachment: signature.asc
Description: PGP signature


Reply to: