[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Finding rough consensus on level of vendoring for large upstreams



Le jeudi 2 septembre 2021, 22:38:35 UTC Phil Morrell a écrit :
> Over this last year there seems to have been a noticeable divergence of
> maintainer opinion, on what has become known as vendoring, from a strict
> reading of [policy 4.13]. I think it's notable that the heading is
> [Embedded] copies and was [Convenience] copies since its inception,
> thankfully I found a request to expand this section using [vendoring].
> 
> [policy 4.13]:
> https://www.debian.org/doc/debian-policy/ch-source.html#embedded-code-copie
> s [Embedded]: https://bugs.debian.org/955036
> [Convenience]: https://bugs.debian.org/392362
> [vendoring]: https://bugs.debian.org/907051
> 
> It is my reading of the situation that not only has this practice become
> more prevalent across multiple ecosystems since 2008, but that it can be
> a good thing when upstreams use it to better modularise their code. As a
> consequence, and in particular for large upstream projects, it is not a
> good use of maintainer time to package every single vendored library as
> a separate source package. See e.g. [kubernetes], [python BoF @25mins],
> [android-platform-tools], and even [uscan] grouping used by nodejs.
> 
> [kubernetes]: https://bugs.debian.org/971515#172
> [python BoF @25mins]:
> https://meetings-archive.debian.net/pub/debian-meetings/2021/DebConf21/debc
> onf21-97-python-team-bof.webm [android-platform-tools]:
> https://salsa.debian.org/android-tools-team/admin/-/issues/40 [uscan]:
> https://manpages.debian.org/uscan#grouped_package

For uscan you misread, the goal of the uscan grouped package is to avoid 
embeded copy. It decouple the concept of upstream package from debian package 
by grouping it, but (see for instance node-resolve unstable package), but 
decreasing the number of embeded copy in the distrib.
- src:node-resolve embded a few related npm package:
   * node-resolve
  * node-types-resolve (the typescript API for this package but that is 
another packages upstream. It is crazy but welcome to nodejs/npm world)
  * is-core-module really small depends upstream of this package
  * path-parse  really small depends upstream of this package

The node-resolve arch:all debian package provides node-type-resolve 
(=version), node-is-core-module (=version) and so on. So apt-get install node-
types-resolve works for user as expected. Moreover other package could depends 
on node-types-resolve (>> version) thus using the normal depends mechanism of 
debian.

Moreover (it is for now contructed by manual rules but me and mabye yadd are 
planning to automate this rules), node-resolve create a /usr/share/doc/node-
types-resolve directory with copyright and changelog.Debian.gz linked back to 
/usr/share/doc/node-resolve/copyright and /usr/share/doc/node-resolve/
changelog.Debian.gz BUT with documentation of node-types-resolve in order to 
allow switching easilly to non grouped package if needed (package being non 
leaf packages or bigger to pass ftpmaster constraint), and in order to let 
your user thinks node-types-resolve is a normal package, with its own 
documentation.

Thus with my pkg-javascript team hat, I use the group feature of uscan, in 
order to deacrease embeded copy in javascript packages and to workarround npm 
one line one package culture.

It is not perfect but it is I believe a good engineering trade of.

The only problem is I am not english native speaker, and thus I have not 
documented why and how we use this feature. Patch to policy are welcome

Moreover I believe that trackers should display the versionned provides and 
https://packages.debian.org/unstable/node-resolve sould also document 
versionned provides and https://packages.debian.org/unstable/node-types-resolve should redirect to d https://packages.debian.org/unstable/node-resolve
in order to avoid to confuse our users.

Bastien


> d https://packages.debian.org/unstable/node-types-resolve
> Is there any objection to the following summary?
> 
> 1. If the reused code is small and intended to be embedded into a
>    package, then this MUST be documented in the [security-tracker].
> 2. If the included project has already been packaged, then the Debian
>    version SHOULD be used instead.
> 3. If modifications have been made, then those changes SHOULD be
>    forwarded and/or the package ported to the official version.
> 4. When 2 or 3 are too onerous to maintain, the package MAY use the
>    convenience copy but MUST document why in README.source and SHOULD be
>    included in the [security-tracker].
> 5. Where only a small number of unrelated projects are bundled, they
>    SHOULD be uploaded as separate source packages.
> 6. If the upstream authors are largely the same, then vendored
>    sub-projects MAY simply be built together as the same source.
> 7. A large number of vendored dependencies used only together for a
>    single Debian package MAY be grouped into a single source upload.
> 8. If 6 or 7 are used initially but a new package has some overlap, then
>    the new package MUST NOT duplicate the vendoring. The duplication
>    SHOULD be packaged separately, then the original package SHOULD be
>    updated to use the Debian version instead.
> 9. When upstream has a proven track record of promptly handling security
>    vulnerabilities inside their vendored dependencies, then maintainers
>    SHOULD follow the same practice, updating versions in lockstep.
> 
> I might be misusing the MUST/SHOULD/MAY, so those can be dropped as
> needed, but I tried to capture the accepted practice and deliberately
> used all the different historical terms. For comparison, there's also
> [Fedora] policy, but apart from not having an in-band "bundled", I also
> think that the line has ended up being drawn marginally differently.
> 
> [security-tracker]:
> https://salsa.debian.org/security-tracker-team/security-tracker/-/blob/mast
> er/data/embedded-code-copies [Fedora]
> https://docs.fedoraproject.org/en-US/packaging-guidelines/#bundling

Attachment: signature.asc
Description: This is a digitally signed message part.


Reply to: