Re: Publishing raw generic{,cloud} images without tar, and without compression, plus versionning of point releases

[Thomas Goirand <zigo@debian.org>]

On 5/27/20 4:50 AM, Ross Vandegrift wrote:
> On Mon, May 25, 2020 at 07:12:15PM +0200, Thomas Goirand wrote:
>> On 5/25/20 5:43 PM, Ross Vandegrift wrote:
>>> I don't think it's obvious how to do better.  The only ways I know to
>>> make a raw image smaller than its fs are:
>>>   1) sparse files
>>>   2) compression
>>>
>>> FAI is using #1, and you want to avoid #2.  Do you know another way?
>>
>> Actually, I do. Shrink the FS once it's prepared, and leave as few space
>> as possible (maybe, 20 GB), then resize the partitions. That way, the
>> final HDD is as small as possible. That's what I was doing optionnally
>> with openstack-debian-images, but I just don't know how this translates
>> for FAI.
> 
> Oh I see- clever.  This could be a nice feature for FAI.  But it should
> also be possible to post-process after FAI is done in the pipeline.
> 
> We'd either need to find the common requirements for all providers (eg AWS
> requires 256MiB free), or limit the reduction to the generic images.
> 
>>> Is avoiding the extra download step more important than reducing the
>>> image size? Your first mail raised both issues, and FWIW, I thought you
>>> were mostly concerned about the size.
>>
>> I'm always bad at explaining, and I need more words than normal people,
>> sorry for this. Let me try again...
>>
>> What's important is reducing the amount of time a user experience. If we
>> provide a raw image file only in the form of a tarball, what's going to
>> happen is that OpenStack users will have to:
>> 1/ Download the image (locally?)
>> 2/ Uncompress
>> 3/ Upload to Glance
>>
>> If that user doesn't have already a VM on the cloud, and is working
>> remotely with a poor upload bandwidth (which is typical with DSL links),
>> uploading to glance is going to take forever, and will be forced,
>> because no other way around it (ie: the archive must be uncompressed
>> before the uplaod).
> 
> This is helpful, thanks - so both matter because they both contribute to how
> long the user must wait.  Though I'd avoid the DSL upload by using my cloud for
> this process... :)

Yes, of course, but this isn't ideal still. It could:
1/ cost you more.
2/ not be possible because there's no Debian image yet (rare cases? :)).
3/ be time consuming as well, because you need to first pop a VM for it.

Multiply this by the number of region you need to maintain...

Cheers,

Thomas Goirand (zigo)