Hi all, Here is some food for thought for the minidebconf that starts tomorrow in Cambridge [1]. Unfortunately I will not make it there, though I wish I did, but I'm in the process of job searching at the moment[2] and could not afford the expense. Anyway, since I got nothing close to a proper reply to my suggestions over IRC or email before, I thought of doing a proper proposal. Keeping the iMX53 locos just doesn't cut it anymore, they're slow and have started failing builds due to lack of RAM (OOM in linking stage for iceweasel [3]), 2 out of 7 are offline, 1 has a failing disk (hoiby, hasse had its own replaced already). And taking ~2 days to see that the build fails is a bit irritating. By contrast, I've built the same package, iceweasel, on my chromebook (dual-core A15) in 11h. I'm using a USB3 external 2.5" case with an SSD disk and the result is a very well performing system. I guess the XU would do the same compile in about 4-6h due to its being 8-core (4xCortex A15 + 4xCortex A7). So I propose something similar: Upgrade 5 of the iMX53s with 5 Odroid-XU and be done with armhf builders for a long time. This is a counter-proposal to what Hector Oron (zumbi) suggested: that we buy or get donations for real server class hardware (arm64). In theory that would be best as it would build all 3 ports, but it has several drawbacks: 1. Terribly expensive: a single board costs ~7k USD. And it needs the chassis which is also expensive. So if you take redundancy in mind, you need 2 setups in 2 locations, so at least 14k just for the boards and I'd not be surprised if the chassis costs more than 5k itself. So in total ~25k. Now if we get this thing donated, even half of it, by all means, that would be great, but the impression I got is that this is unlikely. And I would never propose that Debian spends $25k of its budget for arm* builders. 2. It causes a SPOF (Single Point Of Failure) risk if we go the route of building for all 3 ports on it. If one board breaks, all 3 ports would have to wait for a repair/replacement. Huge downtime which no port can afford. 3. Liability. Right now armhf builders (iMX53 Loco boards) are in ARM HQ and York University, if I'm not mistaken. If something happens and that equipment breaks due to eg. some fault in the electrical wiring and a board burns, the boards are very cheap to replace and there is no issue there. However with $10k+ worth of equipment I'm very doubtful if ARM or York or anyone really would accept to host these systems and accept liability in case of fault. 4. Availability. In short we don't know when those arm64 boards will be available. Most likely 2014Q1 but it's not concrete. So, my counter-proposal to that is that we get instead some cheap easy to replace boards like the Arndale [4] or the Odroid-XU [5]. Personally I'd prefer the XU as it's better equipped, but I'd go with either choice if people think it's better. This way, all 4 points above are solved, we can even buy a couple of spares in each site, in case one breaks down so we can replace it easily. I've done some short list of items, a BOM if you like, converted in GBP since that's where the boards will most likely end up (used amazon for some parts, feel free to look elsewhere, this is just a rough cost estimate): Qty Item Price Total 5 Odroid-XU 105.00 525.00 5 Odroid-XU case 5.61 28.05 5 Odroid-XU serial cable 9.35 46.75 5 Odroid-XU PSU 6.23 31.15 5 2.5" USB3 SATA case [6] 28.50 142.50 5 Kingston 2.5" SSD V300 60GB 41.96 209.80 Total 983.25 GBP So for less than 1k GBP, we can upgrade our complete buildd farm with just 3 builders and keep 2 spares at each site, we could even keep our current iMX53s as spares of course, but seriously why bother? Just keep them for a while and send them to interested developers afterwards (no I'm not interested myself, have enough MX5x hardware already as it is). One issue I didn't mention is how do you solve remote power and serial console management with these boards. This is actually an advantage in favour of the arm64 server class hardware. Well, with a simple google search, I found several products to solve this case but I'm not an expert so I'll let you find the best solution for this. BUT in case anyone is interested I have a PCI Acceleport 8i cart (one PCI multi-serial cart that extends to 8x DB25 ports, not DB9 unfortunately), which I'd be willing to donate if there is a need for this cause -it was actually used in the original armhf EfikaMX buildfarm to monitor each efika. Finally, last but not least, there is the issue of mainline kernel support for exynos5 and the odroid/arndale boards. Right now 3.11 doesn't give you much, but I've heard that the situation with 3.13 is much better -still not completely operational, but it should get better in time. My point with regards to mainline kernel is that it really doesn't matter that much at the moment -yes, I know some will say it matters, but no honestly it doesn't. In a few months time, mainline will get to a working point just like it did with the Locos -for anyone that remembers those used an FSL kernel initially and only moved to mainline a few months afterwards, as soon as the parts we cared about worked well (serial, SATA). I don't see why the same won't be valid for the exynos5 boards. If people *really* don't want non-mainline kernels, then we could go for the utilite boards, I would suggest the Pro version: http://utilite-computer.com/web/utilite-pro-specifications It wouldn't be as fast as the exynos5, but it would be better in terms of networking/SATA as it has dual-gigabit and 32GB SSD, though I guess 32GB SSD might just be enough for a builder. Still, better than the current solution by far. Anyway, apologies for my long email, I really hope you people discuss this issue in the minidebconf the following days and I also hope we will solve this soon, before we get more builders breaking on us. Enjoy and have fun Konstantinos [1] https://wiki.debconf.org/wiki/Miniconf-UK/2013 [2] there are some talks going on but nothing really concrete yet, so if you have an interesting project and looking for remote developers, send me an email :) [3] https://buildd.debian.org/status/fetch.php?pkg=iceweasel&arch=armhf&ver=24.1.0esr-1&stamp=1383943405 [4] http://www.pyrustek.com/ [5] http://www.hardkernel.com/renewal_2011/products/prdt_info.php?g_code=G137510300620 [6] I'm using this one personally, but feel free to choose whichever you feel like http://www.amazon.co.uk/gp/offer-listing/B008PDD6LE/ref=dp_olp_0?ie=UTF8&condition=all [7] Again this is what I use, or its 120GB version, this one would work fine for a builder: http://www.amazon.co.uk/Kingston-Technology-Solid-2-5-inch-Adapter/dp/B00A35X6GM
Attachment:
pgpn2mjnDuBjD.pgp
Description: PGP signature