Arm ports build machines (was Re: Arch qualification for buster: call for DSA, Security, toolchain concerns)
On Fri, Jun 29, 2018 at 11:23:25AM +0200, Julien Cristau wrote:
>On 06/29/2018 09:16 AM, Uwe Kleine-König wrote:
>>>
>>> [DSA Sprint report]:
>>> https://lists.debian.org/debian-project/2018/02/msg00004.html
>>
>> In this report Julien Cristau wrote:
>>
>>> In short, the hardware (development boards) we're currently using to
>>> build armel and armhf packages aren't up to our standards, and we
>>> really, really want them to go away when stretch goes EOL (expected in
>>> 2020). We urge arm porters to find a way to build armhf packages in
>>> VMs or chroots on server-class arm64 hardware.
>>
>> If the concerns are mostly about the hardware not being rackable, there
>> is a rackable NAS by Netgear:
>>
>> https://www.netgear.com/business/products/storage/readynas/RN2120.aspx#tab-techspecs
>>
>> with an armhf cpu. Not sure if cpu speed (1.2 GHz) and available RAM (2
>> GiB) are good enough. The machine can run mainline Linux[1]. I think
>> U-Boot doesn't support this machine in mainline though.
>>
>Rackable, while good, is only part of it. The main part is remote
>management. I'm not seeing any mention of ipmi or anything like that in
>the datasheet?
>
>2G is also way too little memory these days for a new buildd.
Nod - lots of packages are just too big for that now.
So, here's a summary of the current build/porter machines we have for
the arm ports.
armel/armhf *only* build machines
=================================
We're mostly using dev boards to build 32-bit arm stuff yet.
* 7 Marvell Armada XP machines: dev boards supplied in a box, 4GiB
RAM - see http://photos.einval.com/gallery/2014_marvell_buildd
These are nice powerful little machines, but they're a PITA to
manage for power (won't cold boot without a button push) and serial
(USB serial exposed only). We have 6 of these as build boxes and 1
porter box, and I have a spare ready to go into service if
desired. They *don't* have NEON, so we also still have:
* 1 Freescale imx53 dev board as a porter box - old, slow Cortex A8
machine with only 1GiB of RAM. This works better for serial, but
remote power issues again - needs a button push to cold boot. Will
happily retire this once we have NEON available by default.
Each of these dev boards only has support for 1 disk, so disk failures
are painful.
arm64 build machines
====================
These are all more normal computers, with better support for remote
power and serial, DIMM slots, multiple disks, etc.
* APM Mustang (X-Gene 1): officially EOL, but working fine for
now. Normal server-class machine (although supplied in a small
desktop case!) with some onboard server management stuff. We
currently have one of these. We used to have more loaned/hosted by
Linaro, and I've had an offer of more of these if we're
interested. They'll build and run A32 (32-bit instruction set) as
well as A64.
* Gigabyte MP30-AR0 (X-Gene 1): server systems based on the Mustang
core - see
https://b2b.gigabyte.com/Server-Motherboard/MP30-AR0-rev-11#ov
Capable of building/running A32 and A64.
* AMD Seattle (Opteron A1100): officially EOL too, but working
fine. Same as the Softiron 3000, 2U rackmount case. Capable of
building/running A32 and A64. One of these has just been configured
to build armhf only.
Future options
==============
I understand DSA's reluctance to continue supporting dev boards as
build platforms - I've been the one working on some of these machines
in the machine room at Arm, and it's painful when you can't reliably
reboot or get onto the console of crashed machines. We've also had a
spate of disk failures recently which has caused extended downtime.
I'm just in the middle of switching the arm64 machines here to using
SW RAID to mitigate that in future, and that's just not an option on
the dev boards. We want to move away from dev boards for these
reasons, at the very least.
So, at the moment as far as I can see we're happy with our current
arm64 build machines. They are ageing, so obviously I'm continuing to
look out for new options there as well. *But* my priority is new
options for 32-bit building too. Following standard Debian practice,
we want to build *natively* (i.e. not using cross-building or using
hardware emulation). Building 32-bit code on a 64-bit platform should
not be an issue so long as the platform can also execute that 32-bit
code directly.
I am not aware of any 32-bit Arm *server* platforms shipping
today. Some have existed in the past (e.g. Calxeda), but almost
universally people have moved on to 64-bit now. The awkward thing that
is now becoming more common in the arm64 server world is that quite a
few of the vendors are not seeing any value in A32 support so they're
leaving it out of their CPUs. We'll need to be careful about that.
Options I can see today for new publically available machines are
here. I'm sure I'll have missed something obvious - please feel free
to improve on this list!
* Macchiatobin [1] - based on the Marvell 8040 SoC (4-core Cortex
A72), supports both A32 and A64. Standard format (mini-itx) board
mountable in a rack case. DIMM slot supports up to 16GiB, 3 SATA
ports, multiple onboard NICs. Supported in mainline upstream
kernel. Console on USB :-/. Readily available to buy.
* Synquacer [2] - based on the Socionext SC2A11 SoC (24-core Cortex
A53), supports both A32 and A64. Standard format (ATX) board
mountable in a rack case. DIMM slots supports up to 4x16GiB, 2 SATA
ports, onboard NIC. Supported in mainline upstream kernel. I'm
hoping to get some of these machines donated to us from Linaro.
* Qualcomm Centriq [3] - based on Qualcomm's Falkor CPU. Only
supports A64 - no A32 support. All the big features you'd want in a
big expensive server (management, RAM, I/O). Development machines
available, but difficult to get hold of for the general
public. Supported in mainline upstream kernel, some of it
backported for Stretch (9.5) in #896775.
* ThunderX 2 [4] - Cavium's second generation of AArch64 server
CPU. Only supports A64 - no A32 support. All the big features you'd
want in a big expensive server (management, RAM, I/O). Development machines
available, but difficult to get hold of for the general
public. Supported in mainline upstream kernel.
* HiSilicon D05 [5] - HiSilicon's latest AArch64 server CPU and
board. AFAIK only supports A64 - no A32 support. All the big
features you'd want in a big expensive server (management, RAM,
I/O). Development machines available, but difficult to get hold of
for the general public - I've been trying for a while with
HiSilicon. Not sure about upstream kernel support.
While they're on the lower end of this list, I think the Macchiatobin
and Synquacer are probably our best bets *at the moment*, particularly
when considering A32 support. In suitable rack cases with PDU and
serial console, would those work for DSA's needs?
[1] http://macchiatobin.net/
[2] https://www.cnx-software.com/2017/09/24/gigabyte-synquacer-96boards-enterprise-platform-is-powered-by-socionext-sc2a11-24-core-armv8-soc/)
[3] https://www.qualcomm.com/products/qualcomm-centriq-2400-processor
[4] https://www.cavium.com/product-thunderx2-arm-processors.html
[5] http://open-estuary.org/d05/
--
Steve McIntyre, Cambridge, UK. steve@einval.com
Can't keep my eyes from the circling sky,
Tongue-tied & twisted, Just an earth-bound misfit, I...
Reply to: