[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Using Salsa CI with massive source packages



I am trying to enable Salsa CI for the qt6-webengine package.

The package takes a long time to build, longer than the default Salsa 
CI runner timeout of 3 hours.  I learned how to work around this with 
the pyinstaller package by creating my own GitLab Runner running on my 
hardware, which allowed me to specify any length of job timeout.

But with qt6-webengine, I have run into another problem.  Qt6-
webengine has a massive code base (500 GB tarball, 3 TB extracted).  
This leads to 4.3 G of artifacts that try to upload to Salsa after 
extracting the source.  Which fails because it is larger than Salsa’s 
750 MiB limit.

https://salsa.debian.org/qt-kde-team/qt6/qt6-webengine/-/jobs/7400053

My motivation in enabling Salsa CI for qt6-webengine relates to 
efforts to provide better security support, which would benefit from 
Salsa CI.  Information can be found at:

https://salsa.debian.org/qt-kde-team/qt6/qt6-webengine/-/
merge_requests/8

I understand why Salsa needs a limit on the uploading of artifacts.  
My question is if there is any way to work around this for qt6-
webengine.  Is there some way I can store the artifacts on my system, 
similar to how I can implement my own runner?  Can I disable the 
artifacts or cache them locally if other tests need them?  I don’t 
think we generally need the artifacts, just the output displayed for 
each test.

-- 
Soren Stoutner
soren@debian.org

Attachment: signature.asc
Description: This is a digitally signed message part.


Reply to: