DEP18 follow-up: What would be the best path to have all top-150 packages use Salsa CI?
Hi!
In short:
I would very much like to see all top-150 packages run Salsa CI at
least once before being uploaded to unstable. What people think is a
reasonable way to proceed to reach this goal?
Background:
We have had several cases recently where an upload to Debian unstable
causes widespread failure in unstable, and it could have been easily
prevented if Salsa CI pipeline had run for the package and revealed
the problem before upload to archive for everyone to suffer.
This message was prompted by the fact that right now we have a massive
large of Python packages affected by the "No module named 'packaging'"
bug [1] which would have been caught by Salsa CI running the
autopkgtest job before upload [2]. I don't want to blame maintainers
for missing on some details while doing new releases - it happens. But
systematically not using available and easy testing facilities does
annoy me.
We can't have Salsa CI for all of Debian overnight, but at least
widespread issues could be mitigated significantly by having Salsa CI
at least on the top-150 or top-500 packages.
We do not have stats on how many packages use Salsa CI, but we have
stats on how many are not even hosted on Salsa:
```
curl -LO https://trends.debian.net/vcs-hosting_unstable-packages.csv
curl -LO https://popcon.debian.org/sourcemax/by_inst
for x in $(tail -n +13 by_inst | head -n 150 | cut -c 7-25)
do
grep -E "^$x," vcs-hosting_unstable-packages.csv
done | grep -v salsa
dpkg,1.22.10,other
coreutils,9.4-3.1,no vcs
acl,2.3.2-2,other
zlib,1:1.3.dfsg+really1.3.1-1,no vcs
attr,1:2.5.2-1,other
hostname,3.23+nmu2,no vcs
readline,8.2-4,no vcs
e2fsprogs,1.47.1-1,other
base-files,13.3,no vcs
bash,5.2.21-2.1,not git
expat,2.6.2-1,no vcs
gettext,0.22.5-2,no vcs
diffutils,1:3.10-1,no vcs
libbsd,0.12.2-1,other
sqlite3,3.46.0-1,no vcs
dmidecode,3.6-1,other
pciutils,1:3.13.0-1,other
libxdmcp,1:1.1.2-3,git on alioth
wget,1.24.5-2,no vcs
file,1:5.45-3,other
laptop-detect,0.16,other
fuse,2.9.9-8.1,no vcs
lsof,4.95.0-1.1,no vcs
scowl,2020.12.07-2,other
emacsen-common,3.0.5,no vcs
libusb-1.0,2:1.0.27-1,no vcs
setuptools,70.3.0-2,no vcs
traceroute,1:2.1.5-1,no vcs
liblockfile,1.17-1,github
```
If you are a maintainer of a top-150 package and want help in getting
it on Salsa and have Salsa CI activated, just email debian-salsa-ci@,
and we will guide you through how to best run `gbp import-dscs
--debian-branch=debian/latest --upstream-branch=upstream/latest
--pristine-tar`, what to put in a README.source how to activate Salsa
CI with potential customizations you feel are make sense. We have
already done this to many projects, but as surprisingly many
maintainers prefer not to receive contributions, we encourage those
who want to have help to reach out themselves.
When I proposed DEP18[1] my main motivation was to get more packages
on git and on salsa.debian.org so that it is easier to collaborate on
the next upload with the maintainer and all interested contributors
before the upload is done. Collaborating on packages by NMUs is just
not a modern and efficient way to collaborate.
On top of this, I also wished that packages would use Salsa CI, at
least once before the upload. This helps the maintainer get assurance
of the package health before upload. Running Salsa CI on Merge
Requests automatically helps contributors get immediate feedback, and
it also gives the maintainer assurance that contributions don't cause
easily testable regressions.
Many people raised concerns on DEP18, and some of them are valid, and
I will continue to ponder about it and how to restructure the proposal
to foster collaboration without alienating any of our current
maintainers who have a well optimized existing workflow.
However, pushing for wider Salsa CI adoption I think makes sense as a
goal by itself, and I would be keen to hear what people think is a
reasonable way to proceed with that?
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1079175,
https://salsa.debian.org/salsa-ci-team/pipeline/-/issues/376
[2] https://salsa.debian.org/python-team/packages/setuptools/-/jobs/6156348
[3] https://salsa.debian.org/dep-team/deps/-/merge_requests/8
Reply to: