Re: Endianness of data files in MultiArch (was: Please test gzip -9n - related to dpkg with multiarch support)
On Thu, Feb 9, 2012 at 01:35, Simon McVittie <smcv@debian.org> wrote:
> On 08/02/12 17:22, Aron Xu wrote:
>> Some packages come with data files that endianness matters, and many
>> of them are large enough to split into a separate arch:all package if
>> endianness were not something to care about. AFAIK some maintainers
>> are not aware of endianness issues in their packages and then just
>> ignored it (not sure how many, but if any of them are discovered it
>> should lead to RC bug).
>
> Hopefully Jakub Wilk's automatic checks for conflicting files
> <http://people.debian.org/~jwilk/multi-arch/> will already be picking
> this up, in cases where the less-used-endianness architectures aren't
> broken already.
>
> If the less-used-endianness architectures are already broken, that's
> also a bug (potentially an RC one), just like code that compiles but
> doesn't work on a particular endianness due to other assumptions - and
> if nobody has noticed it yet, presumably the package doesn't have any
> users (or regression tests) on those architectures.
>
Or some of them just gave up because it is "less-used" architecture.
>> It would be great to have some mechanism to
>> handle such kind of problems in Debian, to avoid forcing those data to
>> be placed into arch:any package.
>
> If the right endianness is critical: libfoo:i386 Depends:
> libfoo-data-le, libfoo:powerpc Depends: libfoo-data-be, both data
> packages arch:all, data files in /usr/share/foo/le and /usr/share/foo/be
> respectively?
>
This looks not very nice, because we need to maintain a list of
architectures in debian/control, and when new architectures are added
the package is potentially broken.
Also, arch:all packages are usually generated by the uploading DD on
one architecture, mostly amd64 and i386 today, how can he managed to
generate be data files if he doesn't have access to such a machine?
Adding an option to the data generator/parser and make it able to
generate be/le data on any architecture seems not to be a reasonable
approach.
> Or just make sure the data has an endianness marker, and enhance the
> reading package to do the right byteswapping based on the endianness
> marker - e.g. this has been discussed for gettext, which ended up just
> writing out the same endianness on all platforms. Many formats
> (particularly those that originated on Windows) are always
> little-endian, and big-endian platforms reading them just take the minor
> performance hit; formats that respect "network byte order" have the
> opposite situation.
>
This is valid for most-used applications/formats like gettext, images
that are designed to behave in this way, but on the contrary there are
upstream that don't like to see such impact, especially due to the
complexity and performance impact.
Currently I am using arch:any for data files which aren't be affected
with multiarch, i.e. not "same" or "foreign". For endianness-critical
data that is required to make a library working, I have to force them
to be installed into /usr/lib/<triplet>/$package/data/ and mark them
as "Multiarch: same", this is sufficient to avoid breakage, but again
it consumes a lot of space on mirror.
I thought about something like /usr/share/$package/data/{be,le} in
arch:all, but appears to be not a reasonable solution because we need
to modify the data generator/parser.
--
Regards,
Aron Xu
Reply to: