Bioconductor 3.19 transition
Hello everybody,
I would like to prepare the Bioconductor 3.19 transition.
Can we do an atomic transition?
-------------------------------
Ideally, it would be done in an atomic manner, with all packages
uploaded on the day the release team gives us the green light, and then
migration to Testing within the next few days. This ensures that it
does not get entangled with other transitions. There are some obstacles
to this.
1) Some packages may need to go through the NEW queue.
2) The packages with dependency levels N+1 can not be build before
dependency N is built.
3) We have no control on when we get the green light, and if our
packages are built too early, they may miss the previous transitions.
4) Staging the transition in Experimental means that everything has to
be re-uploaded by hand, givent that the Release team can not transition
packages from Experimental to Sid (that is really sad).
Should we just give up instead?
-------------------------------
One solution would be to drop Bioconductor entirely, given the level of
(di)stress that it inflicts to us, the Release team and the FTP team.
To be honest, while I was using our packages in the past, I do not
anymore and just install them from source from time to time in a
container. But that solution does not work: Bioconductor packages are
reverse-dependency of some bioinformatics tool that use them
non-interactively for the user.
Can we automate it so well that a single person can do it?
----------------------------------------------------------
Another solution that I am thinking about would be to automate the
process in a local archive. In Ubuntu it is called a PPA; in Debian we
do not have such a luxury. But if automation works really well, I
wonder if a single person can just do all the work on a local computer…
The advantage of this solution is that packages that are ready can be
uploaded "en masse" as soon as the Release team starts the transition.
Again, packages in Experimental must be re-built because their version
number already exists in the archive.
Can we mass-rebuild Bioconductor packages in user mode?
-------------------------------------------------------
I have asked on debian-devel for tools that allow to be mass rebuild in
user mode. Unfortunatlely it clashes with core invariants of Debian's
nature: there are no easy tools to use apt or dpkg to install a non-root
Debian system in the same way as Miniconda would do, so we need to rely
on sbuild, pbuilder or the like, and they usually require root. I have
been pointed to the usermode capacity of sbuild, but last time I tried
it from a Singularity image, it did not work. This is sad. In user
mode I have access to ~20 Tb of temporary storage and thousands of CPUs
for free…
Can we simulate a transition offline?
-------------------------------------
Let's imagine user-mode building is solved. I see that the transition
tool used by the Release team, "ben", is packaged and may be used on a
local repository. So in principle it is possible to simulate a
transition and report the results to the Release team.
Can we run CI tests offline?
----------------------------
This is the part I have studied the least. Debci is of course open
source. But I do not know if it can run non-root.
Any thoughts?
-------------
If there were the tools to run the rebuilds and CI tests in user mode, I
would be up for preparing the new packages and routine-update the
others. Do you think it is possible. Can you help me for setting up
the usermode operations?
Have a nice week-end,
Charles
--
Charles Plessy Nagahama, Yomitan, Okinawa, Japan
Debian Med packaging team http://www.debian.org/devel/debian-med
Tooting from work, https://fediscience.org/@charles_plessy
Tooting from home, https://framapiaf.org/@charles_plessy
Reply to: