Bug#736097: libc6:amd64: libc6 2.17-97 rm'ing ld.so.cache at inopportune moment, can not upgrade
On Tue, Jan 21, 2014 at 10:26:51AM +0100, Robert Waldner wrote:
>
> On Mon, 20 Jan 2014 23:36:17 -0700, Adam Conrad writes:
> >On Sun, Jan 19, 2014 at 06:55:49PM +0100, Robert Waldner wrote:
> >>
> >> The problem is that as soon as ld.so.cache is gone, dpkg-deb stops working
> >> because it can't find libz.so.1 anymore. At the moment I don't have any
> >> idea on how to upgrade from stable (2.13-38) to testing (2.17-97). Any hints
> >> *greatly* appreciated.
>
> >Your analysis doesn't make much sense to me. If libz.so.1 is on the
> >default search path (and it is, unless you've moved it), you don't
> >need ld.so.cache to exist to resolve it. For things on the search
> >path, the cache only speeds up the lookup, it doesn't facilitate it.
>
> Ok. My knowledge about libc and library-loading isn't great.
>
> >If this weren't true, literally *nothing* would run, because most of
> >the world depends on finding libc.so.6 on path.
> >
> >So, I'd recommend sorting out where your libz.so.1 has gone to, and
> >that should get you going again.
>
> I've prepared a copy of the system for testing via chroot. chroot'ed to
> there, `apt-get upgrade`:
>
> Preparing to replace libc6-dev-i386 2.13-38 (using .../libc6-dev-i386_2.17-97_amd64.deb) ...
> Unpacking replacement libc6-dev-i386 ...
> Preparing to replace libc6-i386 2.13-38 (using .../libc6-i386_2.17-97_amd64.deb) ...
> Unpacking replacement libc6-i386 ...
> Replaced by files in installed package libc6:i386 ...
> Preparing to replace linux-libc-dev:amd64 3.2.51-1 (using .../linux-libc-dev_3.12.6-2_amd64.deb) ...
> De-configuring linux-libc-dev:i386 ...
> Unpacking replacement linux-libc-dev:amd64 ...
> Preparing to replace linux-libc-dev:i386 3.2.51-1 (using .../linux-libc-dev_3.12.6-2_i386.deb) ...
> Unpacking replacement linux-libc-dev:i386 ...
> Can not write log, openpty() failed (/dev/pts not mounted?)
> Setting up linux-libc-dev:amd64 (3.12.6-2) ...
> Setting up linux-libc-dev:i386 (3.12.6-2) ...
> Can not write log, openpty() failed (/dev/pts not mounted?)
> (Reading database ... 288896 files and directories currently installed.)
> Preparing to replace libc6-dev:i386 2.13-38 (using .../libc6-dev_2.17-97_i386.deb) ...
> De-configuring libc6-dev:amd64 ...
> Unpacking replacement libc6-dev:i386 ...
> Preparing to replace libc6-dev:amd64 2.13-38 (using .../libc6-dev_2.17-97_amd64.deb) ...
> Unpacking replacement libc6-dev:amd64 ...
> Preparing to replace locales 2.13-38 (using .../locales_2.17-97_all.deb) ...
> Unpacking replacement locales ...
> Preparing to replace libc6:amd64 2.13-38 (using .../libc6_2.17-97_amd64.deb) ...
> De-configuring libc6:i386 ...
> Checking for services that may need to be restarted...
> Checking init scripts...
> Unpacking replacement libc6:amd64 ...
> dpkg-deb: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory
> dpkg: error processing /var/cache/apt/archives/libc6_2.17-97_amd64.deb (--unpack):
> subprocess dpkg-deb --fsys-tarfile returned error exit status 127
> dpkg-deb: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory
> dpkg: error processing /var/cache/apt/archives/libc6_2.17-97_i386.deb (--unpack):
> subprocess dpkg-deb --control returned error exit status 127
> Processing triggers for man-db ...
> /usr/bin/mandb: error while loading shared libraries: libgdbm.so.3: cannot open shared object file: No such file or directory
> Errors were encountered while processing:
> /var/cache/apt/archives/libc6_2.17-97_amd64.deb
> /var/cache/apt/archives/libc6_2.17-97_i386.deb
> E: Sub-process /usr/bin/dpkg returned an error code (1)
>
> Now, at this point practically nothing works any more:
>
> :( root@fsck->/ # ls -al ./usr/lib32/libz.so.1
> ls: error while loading shared libraries: libselinux.so.1: cannot open shared object file: No such file or directory
> :) root@fsck->/ # find . -name libz.so.1 | xargs file
> file: error while loading shared libraries: libmagic.so.1: cannot open shared object file: No such file or directory
>
> Looking from _outside_ the chroot it seems everything's still there:
> # find /mnt/container/ -name libz.so.1 | xargs file -L
> /mnt/container/usr/lib32/libz.so.1: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, BuildID[sha1]=0x3539dd7120554f14be8c3668d2bad45650192c5f, stripped
> /mnt/container/lib/x86_64-linux-gnu/libz.so.1: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=0x1fb7fe1e239c99d4670d5707a44e723a6752d8e1, stripped
> /mnt/container/lib/i386-linux-gnu/libz.so.1: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, BuildID[sha1]=0x6619298726f5e0d5b293bbd09fc3de8c1e6881f8, stripped
>
>
> Yet, once I run ldconfig:
>
> :( root@fsck->/ # ldconfig
> :) root@fsck->/ # ls -al ./usr/lib32/libz.so.1
> lrwxrwxrwx 1 root root 13 Jan 21 01:33 ./usr/lib32/libz.so.1 -> libz.so.1.2.7
>
> Running dpkg through strace yields:
>
> 23397 open("/lib64/libz.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
> 23397 open("/usr/lib64/libz.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
> 23397 open("/lib/libz.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
> 23397 open("/usr/lib/libz.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
> 23397 writev(2, [{"dpkg-deb", 8}, {": ", 2}, {"error while loading shared libraries", 36}, {": ", 2}, {"libz.so.1", 9}, {": ", 2}, {"cannot open shared object file", 30}, {": ", 2}, {"No such file or directory", 25}, {"\n", 1}], 10) = 117
>
> So it looks for libz.so.1 in /lib64, /usr/lib64, /lib and /usr/lib - in
> none of those directories this is there (see above). So it either
> should be in one of those or one of the directories it actually _is_ in
> should be searched. FWIW:
> # egrep "(x86_64|i386)-linux-gnu" /etc/ld.so.conf.d/*
> /etc/ld.so.conf.d/x86_64-linux-gnu.conf:/lib/x86_64-linux-gnu
> /etc/ld.so.conf.d/x86_64-linux-gnu.conf:/usr/lib/x86_64-linux-gnu
> /etc/ld.so.conf.d/yy_i486-linux-gnu.conf:/lib/i386-linux-gnu
> /etc/ld.so.conf.d/yy_i486-linux-gnu.conf:/usr/lib/i386-linux-gnu
>
> LD_LIBRARY_PATH is not set in the environment, so I think (this is a
> completely new frontier for me) it _should_ search in the correct
> places.
>
> For lack of actual clue I now tried:
>
> LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/lib/x86_64-linux-gnu dpkg -i /var/cache/apt/archives/libc6_2.17-97_*
> (Reading database ... 288963 files and directories currently installed.)
> Preparing to replace libc6:amd64 2.13-38 (using .../libc6_2.17-97_amd64.deb) ...
> Checking for services that may need to be restarted...
> Checking init scripts...
> Unpacking replacement libc6:amd64 ...
> Preparing to replace libc6:i386 2.17-97 (using .../libc6_2.17-97_i386.deb) ...
> Unpacking replacement libc6:i386 ...
> Setting up libc6:amd64 (2.17-97) ...
> Checking for services that may need to be restarted...
> Checking init scripts...
>
> Restarting services possibly affected by the upgrade:
> spamassassin: restarting...done.
>
> Services restarted successfully.
> Setting up libc6:i386 (2.17-97) ...
> /bin/sh: error while loading shared libraries: __vdso_time: invalid mode for dlopen(): Invalid argument
> dpkg: error processing libc6:i386 (--install):
> subprocess installed post-installation script returned error exit status 127
> Errors were encountered while processing:
> libc6:i386
>
>
> Looks better, but still leaves me with a hosed system, just different:
>
> :) root@fsck->/ # LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/lib/x86_64-linux-gnu dpkg -i /var/cache/apt/archives/libc6_2.17-97_*
> dpkg: error while loading shared libraries: __vdso_time: invalid mode for dlopen(): Invalid argument
>
> Ok, start from scratch.
>
> This time I ran apt-get upgrade with
> LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/lib/x86_64-linux-gnu:/lib/i386-linux-gnu
>
> And while the upgrade didn't run smoothly (problem with locales),
> libc6_2.17-97* installed fine, and the rest could be dealt with via
> `apt-get -f install` (without LD_LIBRARY_PATH).
>
> So, I guess (guess!=know) the real problem is that for some reason
> something doesn't look in the configured library paths.
The problem is that you have libc6-amd64:i386 installed on your amd64
system, in addition to the system one libc6:amd64 one. This doesn't
bring anything to your system (except bugs like this), but the multiarch
specification doesn't provide a way to prevent such a package to be
installed.
In your case, the problem is that ldconfig changes the
/lib64/ld-linux-x86-64.so.2 to point to the libc6-amd64:i386 version,
so the wrong libc is used on your system. When you upgrade, you start
to have a mix of both libc, hence the issue. We are currently working on
a solution that allow libc6-amd64:i386 to be installed without breaking
the system.
For already broken systems like yours, here is a procedure to fix it:
- make sure ldconfig won't be run anymore:
ln -sf /bin/true /lib/ldconfig
- point the symlink to the correct libc version:
ln -sf /lib/x86_64-linux-gnu/libdl-2.17.so /lib64/ld-linux-x86-64.so.2
note that the version 2.17 might have to be adjusted depending on the
libc currently installed on your system
- remove libc6-amd64:
apt-get remove libc6-amd64
- reinstall and/or upgrade at least libc6 and libc-bin:
apt-get --reinstall libc6 libc-bin
If you have a system which doesn't boot anymore, the two first steps
can be done on a rescue system, and the two other later after booting
on the system.
--
Aurelien Jarno GPG: 1024D/F1BCDB73
aurelien@aurel32.net http://www.aurel32.net
Reply to: