Re: Broadinstitute catch - elegant way to disable tests requiring online access?
On 28.07.21 14:27, Nilesh Patra wrote:
Hi Steffen,
On 28/07/21 01:06 PM, Steffen Möller wrote:
Hi Nilesh,
The import of pristine-tar has worked after removing .gitattributes, but
then the git lfs references were still in the tarball and the
upstream branch. pristine-tar could be pushed, but then the other branches would
trigger git lfs errors when pushed to salsa. Only after having all fasta.gz
lfs files removed, the upload went smoothly and you all now find this on
https://salsa.debian.org/med-team/broad-catch
Is that really intended?
We would not be able to run tests in this case since you essentially
ended
up repacking it , since several tests seem to be using that data.
BTW, I tried to do a little solution for the lfs thingy, for it to not
store "references" and committed it to my personal repository[1]
Can you have a look and let me know if it looks sensible?
Also, you might as well want to have a look at pristine-lfs[2] which
could
be interesting to use. I've attempted to use this too, please consider
taking a look
I admit, I'm not very used to the lfs workflow, so something could be
wrong
for sure.
Not sure why the CI fails though -- probably it does not work fine with
lfs, but just thinking of it as a ground for more ideas here :)
I can clone it locally in a different space and re-produce the tarball,
and
I fixed a few tests (not all) -- please consider trying once and let me
know
[Edit]: all tests pass now on a clean chroot for me
You have outperformed yourself on this one. Thank you tons!
I just removed the broad-catch repository of mine from salsa, please
kindly inject what you have, instead. I am saying that also since
apparently my SALSA_TOKEN apparently does not have authority to adjust
the CI settings.
Pushed to med-team salsa
https://salsa.debian.org/med-team/broad-catch
May I also ask you to upload the package?
I actually wanted to ask a few things before that:
0) Are my changes even correct - is this the way out?
This is my first time packaging a lfs based repository
No idea if it is _the_ way out - but it was _a_ way out at least that
allowed the package to test.
1) When I did the changes, I realised then that the size of repository
goes really, really huge
$ du -sh broad-catch
785M broad-catch
I did not anticipate the package to grow _that_ much. Without the lfs
data it was just at 22M.
(well, a good portion of this is due to both pristine-tar and
pristine-lfs branches)
So do we really want to go that way? I was initially of the impression
that it would be a few megs, but that's quite unfortunately not the case
We can alternatively go with what you pushed earlier and disable tests
instead
2) If we intend to upload this as is, then we really, _really_ need to
either remove installation of all the example fasta files, or we need to
do a separate -examples binary package
Yes. Or get a solution with getData as a test-dependency. But we would
not want to run this with every commit.
3) Would FTP masters be happy with this? Is such a large size OK to go
into the archive?
I first read this a "zero K" :)
The data they ship was last updated in 2018 if I read this right. In
that sense it is nothing volatile.
If we had our own Debian Med repository then this would all be a
no-brainer. We would upload the whole thing or just the data to it.
4) Is the package *that* useful to do that sort of solution for the
same?
No. And even if. Heck, let the folks install it themselves :)
5) Is it easy to long-term maintain it in the current state?
I admit, my connection would not be so reliable to upload all of it in
one shot - atleast not today.
@Andreas, can you take over the uploading work?
And would you also chime into the discussion here?
Next steps would be tell routine-update not to try updating this one. I
can do that.
I think uscan will work fine on this. The next step would be to
introduce pristine-lfs support instead, I guess?
I need to educate myself on pristine-lfs.
And then I would like someone from upstream to comment on
this package and direct us a bit on what else would be good to have in
Debian to help their cause. My "plan" about that is to just go the
official route and introduce the package in a github issue - if it is a
regular user of that package replying, I guess we are just about as happy.
Yes, that sounds sensible
However, I mightn't have a lot of time to spend over this
Maybe we should just leave it in salsa for now and wait if there is any
demand for that software surfacing on the mailing list. The issue github
we can also post with the work residing in salsa.
Best,
Steffen
Reply to: