Re: Publishing raw generic{,cloud} images without tar, and without compression, plus versionning of point releases

[Bastian Blank <waldi@debian.org>]

On Tue, May 26, 2020 at 10:29:17AM +0200, Thomas Goirand wrote:
> > The images have 700-800MB of space used, which is still three times the
> > size of the qcow2 file.  Why do you refuse to use what's already there?
> 
> I already explained: because downloading from ADSL, extracting, then
> re-uploading to a cloud provider, is a stupid way of doing things, when
> one could just point to the URL of the artifact and let the cloud
> provider do the download.

Yes, but you point it to the _qcow2_ image.  Which, for generic, is
about 250-300MB in size, as it is somewhat compressed, and can be read
by all the OpenStack tools.

(Fun fact: even the OpenStack provided tool to generate the Octavia
images does qcow2 by default.)
[https://docs.openstack.org/octavia/latest/admin/amphora-image-build.html]

> Also, as I wrote earlier, it's not 700-800MB, but less than 512MB, and
> we could reduce the size of the raw images to that.

Please show me this existing 512MB image you are talking about.  At
least right now it does not exist.  The build log currently even clearly
states how much space is used.

At the last sprint we talked about how minimal the image should be.  The
majority wanted to have more tools in it.

> >>>> One thing though: if I understand well, artifacts are first stored on
> >>>> Salsa, and currently, there's a size limit. What is the max size? If I'm
> >>>> not mistaking, it's 1GB max, right? If that's the case, then maybe
> >>>> that's a problem with the current 2GB decompressed disk.raw image.
> >>> It's 250MB.
> >> Then how are the ppc64el images generated? (they are bigger than this)
> > No, they are not.  It's just 180MB:
> > | $ xz -vk5T0 *.tar
> > | debian-sid-generic-ppc64el-official-20200525-274.tar: 184.0 MiB / 889.7 MiB = 0.207, 8.1 MiB/s, 1:49
> > (https://salsa.debian.org/cloud-admin-team/debian-cloud-images-daily/-/jobs/762303)

You explicitly asked about Salsa and for Salsa my answer is correct.

> That's not the location I had in mind, but this one:
> http://cdimage.debian.org/cdimage/cloud/buster/20200511-260/
> In there, debian-10-generic-ppc64el-20200511-260.qcow2 (for example) is
> more than 256 MB. How was it generated? Directly on Casulana?

This file is never stored as artifact on Salsa.  Maybe this log will
enlighten you how it is created and stored:

https://salsa.debian.org/cloud-admin-team/debian-cloud-images-daily/-/jobs/765786

Regards,
Bastian Blank

-- 
Spock: We suffered 23 casualties in that attack, Captain.