Bug#882255: libc6-amd64: Multilib causes catastrophic system failure during upgrade to libc 2.25
On Tue, 21 Nov 2017, Aurelien Jarno wrote:
> On 2017-11-21 01:01, Mikulas Patocka wrote:
> >
> >
> > On Tue, 21 Nov 2017, Aurelien Jarno wrote:
> >
> > > On 2017-11-21 00:12, Mikulas Patocka wrote:
> > > >
> > > >
> > > > On Mon, 20 Nov 2017, Aurelien Jarno wrote:
> > > >
> > > > > On 2017-11-20 19:13, Mikulas Patocka wrote:
> > > > > > Package: libc6-amd64
> > > > > > Version: 2.25-1
> > > > > > Severity: critical
> > > > > > Justification: breaks the whole system
> > > > > >
> > > > > > Dear Maintainer,
> > > > > >
> > > > > > *** Reporter, please consider answering these questions, where appropriate ***
> > > > > >
> > > > > > * What led up to the situation?
> > > > > >
> > > > > > I have a x86-64 system with i386 and x32 foreign architectures (because I
> > > > > > need to develop software for i386 and x32 architectures).
> > > > > >
> > > > > > * What exactly did you do (or not do) that was effective (or
> > > > > > ineffective)?
> > > > > >
> > > > > > I ran apt update and apt upgrade. Apt tried to upgrade to libc-2.25.
> > > > > >
> > > > > > * What was the outcome of this action?
> > > > > >
> > > > > > Halfway through apt upgrade it failed and I ended up with unusable system where
> > > > > > large number of binaries were segfauting on startup without doign anything.
> > > > > >
> > > > > > * What outcome did you expect instead?
> > > > > >
> > > > > > The upgrade to libc-2.25 should work.
> > > > > >
> > > > > >
> > > > > > The reason for the catastrophic failure is this:
> > > > > >
> > > > > > There is package libc6-amd64:i386 and libc6-amd64:x32 (which provide
> > > > >
> > > > > I guess you mean you have installed one of the two, not both.
> > > > >
> > > > > > x86-64 libc in /lib64/). This package is not technically needed (because
> > > > > > x86-64 libc is already installed in /lib/x86_64-linux-gnu/), but it is
> > > > > > installed nonetheless because of some dependencies.
> > > > >
> > > > > Just to be clear, as said in my other email, this *is* technically
> > > > > needed as gcc-multilib is not able to make use of the libc in
> > > > > /lib/x86_64-linux-gnu.
> > > > >
> > > > > > apt makes sure that all libc packages are upgraded at once to the same
> > > > > > version. However, during the upgrade process, the package
> > > > > > libc6-amd64 is upgraded before libc6:amd64. So, during the upgrade, we
> > > > > > temporarily have two libcs with different versions on the system, and this
> > > > > > mismatch makes most of the x86-64 binaries crash. Due to the crashes, the
> > > > > > upgrade doesn't proceed and it doesn't install the correct libc version in
> > > > > > /lib/x86_64-linux-gnu/.
> > > > > >
> > > > > > The result is unusable system.
> > > > >
> > > > > I have done some tests, and while I confirm that libc6-i386:amd64 is
> > > >
> > > > The problem is with libc6-amd64:i386 or libc6-amd64:x32.
> > > > Not libc6-i386:amd64.
> > >
> > > Yes, sorry about that, I really did the test with libc6-amd64:i386, but
> > > mixed it when typing the mail.
> > >
> > > > I.e. use amd64 Debian Sid base installation, add foreign architectures
> > > > i386 and x32 and use this /etc/apt/sources.list:
> > > >
> > > > deb [ arch=i386,amd64 ] http://ftp.cz.debian.org/debian/ sid main contrib non-free
> > > > deb [ arch=x32 ] http://ftp.de.debian.org/debian-ports/ unreleased main
> > > > deb [ arch=x32 ] http://ftp.de.debian.org/debian-ports/ unstable main
> > > >
> > > > > unpacked much before libc6:amd64, it doesn't cause any issue here.
> > > > > Indeed the search path in ld.so is to give higher priority to
> > > > > /lib/x86_64-linux-gnu/ over /lib64. I have even been able to install
> > > > > libc6:amd64 in version 2.24 (using force-all), while keeping
> > > > > libc6-amd64:i386 in version 2.25.
> > > > >
> > > > > The only way to change the priority of the two path is using a
> > > > > non-standard ld.so.conf. Have you made any change to /etc/ld.so.conf or
> > > > > /etc/ld.so.conf.d/*? Maybe you can share the content of this file and
> > > > > this directory.
> > > >
> > > > On my system, there's a file /etc/ld.so.conf.d/zz_amd64-biarch-compat.conf
> > > > containing:
> > > >
> > > > # Legacy biarch compatibility support
> > > > /lib64
> > > > /usr/lib64
> > > >
> > > > and /etc/ld.so.conf.d/zz_i386-biarch-compat.conf containing:
> > > >
> > > > # Legacy biarch compatibility support
> > > > /lib32
> > > > /usr/lib32
> > > >
> > > > These files are created by the packages libc6-i386:x32 and
> > > > libc6-amd64:x32. They cause that /lib64 is preferred to
> > > > /lib/x86_64-linux-gnu/. If I delete these files and run ldconfig, the
> > > > linker will prefer /lib/x86_64-linux-gnu/.
> > >
> > > Those files are installed with the zz_ prefix to make sure they are
> > > included last, and therefore after x86_64-linux-gnu.conf. It seems to
> > > be missing on your system and is probably the root of your problem. This
> > > file is installed by libc6:amd64.
> > >
> > > Aurelien
> >
> > I have /etc/ld.so.conf.d/x86_64-linux-gnu.conf, it contains
> > # Multiarch support
> > /lib/x86_64-linux-gnu
> > /usr/lib/x86_64-linux-gnu
> >
> > But nonetheless, the linker prefers libc from /lib64.
>
> You mean that even now running 'ldconfig' followed by 'ldd /bin/true'
> will give you
> libc.so.6 => /lib64/libc.so.6
> instead of
> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
> ?
Yes.
# ldd /bin/true
linux-vdso.so.1 (0x00007ffc62b5a000)
libc.so.6 => /lib64/libc.so.6 (0x00007fd870377000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd870921000)
If I use the --inhibit-cache flag, it points to the correct location:
# /lib64/ld-linux-x86-64.so.2 --list --inhibit-cache /bin/true
linux-vdso.so.1 (0x00007ffd566e4000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f73026c8000)
/lib64/ld-linux-x86-64.so.2 (0x00007f7302c73000)
Without --inhibit-cache, it points to /lib64/libc.so.6
# /lib64/ld-linux-x86-64.so.2 --list /bin/true
linux-vdso.so.1 (0x00007ffd5ad10000)
libc.so.6 => /lib64/libc.so.6 (0x00007fc566092000)
/lib64/ld-linux-x86-64.so.2 (0x00007fc56663c000)
> Also where he symlink /lib64/ld-linux-x86-64.so.2 points to? It should
> point to /lib/x86_64-linux-gnu/ld-2.25.so.
Yes, it points to /lib/x86_64-linux-gnu/ld-2.25.so
> > I have these files in /etc/ld.so.conf.d:
> >
> > i386-linux-gnu.conf:# Multiarch support
> > i386-linux-gnu.conf:/lib/i386-linux-gnu
> > i386-linux-gnu.conf:/usr/lib/i386-linux-gnu
> > i386-linux-gnu.conf:/lib/i686-linux-gnu
> > i386-linux-gnu.conf:/usr/lib/i686-linux-gnu
> > libc.conf:# libc default configuration
> > libc.conf:/usr/local/lib
> > x86_64-linux-gnu.conf:# Multiarch support
> > x86_64-linux-gnu.conf:/lib/x86_64-linux-gnu
> > x86_64-linux-gnu.conf:/usr/lib/x86_64-linux-gnu
> > x86_64-linux-gnux32.conf:# Multiarch support
> > x86_64-linux-gnux32.conf:/lib/x86_64-linux-gnux32
> > x86_64-linux-gnux32.conf:/usr/lib/x86_64-linux-gnux32
> > zz_amd64-biarch-compat.conf:# Legacy biarch compatibility support
> > zz_amd64-biarch-compat.conf:/lib64
> > zz_amd64-biarch-compat.conf:/usr/lib64
> > zz_i386-biarch-compat.conf:# Legacy biarch compatibility support
> > zz_i386-biarch-compat.conf:/lib32
> > zz_i386-biarch-compat.conf:/usr/lib32
> >
> > /etc/ld.so.conf contains just one line:
> > include /etc/ld.so.conf.d/*.conf
>
> That looks correct.
>
> Aurelien
>
> --
> Aurelien Jarno GPG: 4096R/1DDD8C9B
> aurelien@aurel32.net http://www.aurel32.net
>
Reply to: