Re: Debian/ppc64el feasiability to become an official architecture
Hi Peter,
Thank you for your reply.
On 07/24/2014 08:14 PM, peter green wrote:
> Note: this is the perspective of a dd who is not directly involved with powerc
> though I have come across some of your bug reports, nor am I a member of the ftp
> or release teams. It's probablly mostly right but i'm sure others will point out
> any errors.
>
>> I would like to share the ppc64el port's status with you, and check if
>> it is feasible to consider it as an official port for the next Debian
>> release, or, what it may be missing for that.
> quite a bit needs to be done and I personally think it's unlikely any new
> architectures will make it in time for the jessie freeze at this point.
>
> The first step is going to be persuading the ftp masters to let you into the
> debian archive. You can see the ftpmasters critera at
> https://ftp-master.debian.org/archive-criteria.html .
Right, as I understand we are on track to meet all those criteria. We have a good
archive coverage, a debian installer, fast machines doing the build, and, finally
a DD helping us.
> It sounds like the main
> thing you will need to do to meet them is push your fixes into debian proper so
> ppc64el can be built without external patches.
Right?.What do you mean by built? We have a huge amount of packages building for
ppc64el without external patches. At this time, we already built more than 8k8
source architecture-dependent packages (of almost 10K) for this platform[1]
Also, the debootstrap packages are all BFS on ppc64el, and if you want a
environment that contains git, openjdk, ssh, vim and latex, less than 10 bugs need
to be accepted[2]
[1] http://ftp.unicamp.br/pub/ppc64el/debian/buildd-upstream/build_logs/Uploaded.html
[2] https://wiki.debian.org/ppc64el/topBugs
So, I understand that we build a whole amount of things without any external patches.
Merely submitting patches to the
> BTS for issues specific to your architecture is not sufficient. You will almost
> certainly have to perform non-maintainer uploads (NMUs) to deal with unresponsive
> maintainers. While in principle you can ask for sponsorship for NMUs if you want
> to actually get things moving in a reasonable timeframe you will need at least one
> Debian Developer (DD) on your team to upload them.
Right. We have been involved and with a lot of NMU helps. Also, Adam (who is the
DD) kindly agreed to help this architecture. He is rebuilding everything from
source using his key, so, the packages become signed by a DD, and it might be
imported by FTP master.
> You will also need at least one DD on the port team to satisfy the ftpmaster
> critera. Ideally you want more.
>
> Another question the ftpmasters will likely have is what is the relationship
> between ppc64 and ppc64el. Is there hardware that will run ppc64 but not ppc64el?
> is there hardware that will run ppc64el but not ppc64? is there hardware that will
> run both? what is the relative prevalance of each of these. If most hardware that
> can run ppc64el can also run ppc64 then are there significant technical advantages
> other than compatibility with badly written software of going little endian?
Right, at the moment, all new hardware that can run ppc64el can also run ppc64.
The current Power machines are bi-endian, so, you can run both architectures in
the same hardware.
At the same time, they are not compatible, since the ppc64el is based on a fresh
new ABI, while the ppc64 and powerpc uses the old ABI. So, you will not be able to
do a multi-arch between ppc64/powerpc and ppc64el.
Also, ppc64el started with the POWER8 only processor, and the compiled packages
might not be compatible with the old ppc64/powerpc hardware ISA.
> It's not strictly a requirement but it would likely help immensely to get the
> architecture on debian-ports.org so that maintainers can easilly see if their
> packages are failing and porters for other new ports (i've been helping out a bit
> with arm64 myself) can see if ppc64el is also failing.
Right. Ppc64el is pursuing to get into debian-ports since the beginning of the
year, but, there is lack of resources (CPU and memory IIRC) on debian-ports
machines, so, that is the motivation that we didn't get into debian-ports. The
same is happening with other architectures, AFAIK. I have been working with the
debian-ports folks, and as soon as the lack of resources get sorted out, we might
move our arch to debian-ports. Meanwhile, we have our own buildd and
wanna-build[3] that is in sync with every new package that arrives at Debian,
meaning that we are building every single package that show up in the unstable
archive.
[3] http://ftp.unicamp.br/pub/ppc64el/debian/buildd-upstream/build_logs/
> If and when the ftpmasters decide to include your architecture a minimal set of
> packages will be imported and everything (including when possible the minimal set)
> must be rebuilt. This will take time and will likely involve more NMUs. It is
> likely you will need to NMU to fix issues that are not strictly related to your
> port but are nevertheless blocking your build.
Right. We are kind of doing this for some time
> Once you are in the official archive and have rebuilt (nearly) everything you need
> to convince the release team that you meet the release critera. You can find the
> full details at https://release.debian.org/wheezy/arch_policy.html but the main
> ones are you have to have built almost all of debian*, you have to have a working
> installer, you have to actually have users and developers of the debian port, you
> have to be keeping up (which means both having plenty of build hardware AND
> stomping on architecture specific build failures)
Right. We are working hard to solve the package coverage, as I said, we have more
then 300 BTS patches submitted and waiting for maintainers, and almost 90%
architecture-dependent source packages already build for this platform, and, as
soon as the patches get accepted, this number will increase even more.
Regarding the debian installer, there are some patches sent to add ppc64el into
the d-i, and the are still having some discussions about it, but, I understand
that we are on track of it.
Regarding users, this is one good concern, since, the architecture is quite new,
and to be sincere, we don't have a lot of users other than developers, mainly
because ppc64el is still hosted by ourselves (at Unicamp)[4], so, it is hard to
have users if we don't get a more official mark for the architecture.
[4] http://ftp.unicamp.br/pub/ppc64el/
> When you are added to testing you will be added as a "broken and fucked" (release
> team's terminology not mine) architecture. To get out of this state you will need
> to get and keep your port in a healthy state in testing. That will mean fixing (in
> some cases through NMUs) issues that are blocking migration of packages you need
> (whether or not those issues are related to your architecture) and fixing any
> architecture specific build failures as quickly as possible (since when you are in
> the "broken and fucked" state your builds will not be blockers for testing
> migration so a new upload that breaks your architecture will be able to migrate).
Right. Do you recommend us to start building packages from 'testing' right now?
We can create another buildd instances that only build testing and we can see how
healthy it is.
To be fair, this is a good point that I will be focused on the next days.
> Once your port reaches a healty state in testing (most packages present, virtually
> no packages outdated, very few architecture specific packages uninstallable), the
> release team will remove the "broken and fucked" status and you will become a
> release architecture. You then need to maintain your port in a healthy condition
> until release time.
Right.
> *the release team say 98% but that number excludes certain poorly defined
> categories of package and so there is no easy way to measure it. Going by the
> statss on buildd.debian.org (which include everything including packages that are
> fundamentally architecture specific) the lowest release linux architecture
> according is currently at about 96% while kfreebsd is at about 90%. Realistically
> you probablly need to at least match the lowest release linux architecture.
Nice. I was working originally working around 90% of archive coverage, but, maybe
I should leverage this number to around 96%. Well, I continue to think that this
is feasible, at the rate of packages that we have at the moment.
Thank you very much for the good discussion and the details on the process,
Breno
Reply to: