Re: "CD/DVD/Blue-Ray image building expected to take around 8 hours end to end"
Hey Hideki,
On Mon, Jun 17, 2013 at 10:34:13PM +0900, Hideki Yamane wrote:
>On Sun, 16 Jun 2013 02:26:01 +0100
>Steve McIntyre <steve@einval.com> wrote:
>> Building a (potentially large) set of images for an individual
>> architecture takes ~30-40 minutes end-to-end; we currently build for
>> the following 15 arches:
>>
>> i386 source amd64 multi-arch powerpc armel armhf ia64 mips mipsel s390
>> s390x sparc kfreebsd-amd64 kfreebsd-i386
>
> Is it built with multiple hosts (for each arch) or single host?
They're all built on one big server machine (pettersson.d.o).
>> That's where the time goes. The sets we're building for each arch can
>> include up to 9 different types of image:
>>
>> netinst
>> netinst with firmware
>> normal full CD set (Gnome desktop)
>> kde CD
>> lxde CD
>> xfce CD
>> full DVD set
>> full BD set
>> full DL-BD set
>>
>> They're currently all run in parallel as we build for one
>> architecture, getting a massive win from paralellism and caching on
>> pettersson (the CD build machine). I'm hoping we can add some SSDs
>> into pettersson next, to make the build process even faster.
>
> Do those 9 images run at once?
Yes, they do. We're running on a large RAID with lots of spindles in
RAID-10 to maximise random-access throughput.
> Surely SSD makes it faster a bit, but I recommend to use PCIe flash.
> I've tested debian-cd script for amd64 arch with HDD/SSD(raid0)/ioDrive(raid0),
> archive data on each 1 partition and output to same partition.
>
>-------------------------------------------------------------------
># make image-trees
>
>HDD
>
>real 20m34.608s
>user 4m47.538s
>sys 7m32.480s
>
>
>SSD
>real 13m58.332s
>user 4m43.626s
>sys 7m57.726s
>
>
>ioDrive
>real 12m49.472s
>user 4m46.786s
>sys 8m50.189s
>
>------------------------------------------------------------
># make images (normal full CD set)
>
>HDD
>
>real 68m4.551s
>user 30m41.327s
>sys 3m55.095s
>
>SSD
>
>real 50m4.132s
>user 30m31.302s
>sys 3m51.378s
>
>
>ioDrive
>real 19m2.295s
>user 29m59.704s
>sys 3m42.222s
>------------------------------------------------------------
>
> Probably we don't want to use ioDrive since it needs proprietary driver,
> but similar PCIe flash device (Intel 910?) would be able to gain better
> performance than normal SSD drives.
Possibly, yes. We'd need a lot of space, on the order of 2TB to make
things work sensibly.
--
Steve McIntyre, Cambridge, UK. steve@einval.com
"I suspect most samba developers are already technically insane... Of
course, since many of them are Australians, you can't tell." -- Linus Torvalds
Reply to: