Bug#961128: apt-transport-https: https fails with segmentation fault
Dear Maintainer, hello Mehturt,
I guess the missing debug information is contained in libcurl3-dbg [1].
I tried to find out to which line this address in Mehturt's backtrace
points to and came to this location:
(gdb) bt
#0 0x00007ffff77cd4d7 in Curl_close (data=0x55555579f6c0) at url.c:399
#1 0x000055555555874b in HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:538
#2 main (argv=<optimized out>) at ./methods/https.cc:546
https://sources.debian.org/src/curl/7.52.1-5+deb9u10/lib/url.c/#L399
https://sources.debian.org/src/apt/1.4.10/methods/https.cc/#L538
In my opinion the instruction "mov 0x60(%rbx),%rdi" is executed, for Mehturt,
while $rbx contains no valid pointer.
The crash did not show up in my test VM. Attached an example debug session
to debug to this location.
@Mehturt: maybe you can still install libcurl3-dbg and inspect the core again.
(Maybe adding a "display/i $pc" and "print/x $rbx", before any "bt full")
Maybe related, upstream has added a null pointer check to Curl_close.
https://sources.debian.org/src/curl/7.68.0-1/lib/url.c/#L325
Kind regards,
Bernhard
[1] https://snapshot.debian.org/package/curl/7.52.1-5%2Bdeb9u10/#libcurl3-gnutls_7.52.1-5:2b:deb9u10
# Stretch/oldstable amd64 qemu VM 2020-05-23
#sources.list
deb http://192.168.178.25:9999/debian-9-stretch-debug.mirrors.debian.org/ stretch-debug main
deb http://192.168.178.25:9999/debian-9-stretch-debug.mirrors.debian.org/ stretch-proposed-updates-debug main contrib non-free
#approx.conf
debian-9-stretch-deb.debian.org http://deb.debian.org/debian/
debian-9-stretch-security.debian.org http://security.debian.org/
debian-9-stretch-debug.mirrors.debian.org http://debug.mirrors.debian.org/debian-debug/
# or https://wiki.debian.org/AutomaticDebugPackages
apt update
apt dist-upgrade
apt install systemd-coredump
# https://github.com/matrix-org/synapse/blob/master/INSTALL.md
apt install -y lsb-release wget apt-transport-https
wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main" | tee /etc/apt/sources.list.d/matrix-org.list
apt update
apt install matrix-synapse-py3
# no crash
root@debian:~# dpkg -S /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4
libcurl3-gnutls:amd64: /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4
root@debian:~# dpkg -l | grep libcurl3-gnutls
ii libcurl3-gnutls:amd64 7.52.1-5+deb9u10 amd64 easy-to-use client-side URL transfer library (GnuTLS flavour)
apt install gdb apt-transport-https-dbgsym libcurl3-dbg
rm matrix-synapse-py3_1.13.0+stretch1_amd64.deb
gdb -q
set width 0
set pagination off
file /usr/bin/apt
set detach-on-fork off
set follow-fork-mode child
set follow-exec-mode new
b HttpsMethod::~HttpsMethod
y
run download matrix-synapse-py3
info inferiors
inferior 1
cont
info inferiors
inferior 1
set detach-on-fork on
cont
info inferiors
bt
cont
stepi
stepi
benutzer@debian:~$ rm matrix-synapse-py3_1.13.0+stretch1_amd64.deb ^C
benutzer@debian:~$ gdb -q
(gdb) set width 0
(gdb) set pagination off
(gdb) file /usr/bin/apt
Reading symbols from /usr/bin/apt...(no debugging symbols found)...done.
(gdb) set detach-on-fork off
(gdb) set follow-fork-mode child
(gdb) set follow-exec-mode new
(gdb) b HttpsMethod::~HttpsMethod
Function "HttpsMethod::~HttpsMethod" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (HttpsMethod::~HttpsMethod) pending.
(gdb) run download matrix-synapse-py3
Starting program: /usr/bin/apt download matrix-synapse-py3
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New process 3044]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Reading symbols from /usr/lib/debug/.build-id/77/5143e680ff0cd4cd51cce1ce8ca216e635a1d6.debug...done.
Reading symbols from /usr/lib/debug/.build-id/db/2caeeec37482a98ab1416d0a9afe2944930de9.debug...done.
Reading symbols from /usr/lib/debug/.build-id/ea/d5fd817712e63c1212d1ee7d7ee1b9c29f93a7.debug...done.
Reading symbols from /usr/lib/debug/.build-id/16/d609487bcc4acbac29a4eaa2dda0d2f56211ec.debug...done.
Reading symbols from /usr/lib/debug/.build-id/4e/49714c557ce0472c798f39365ca10f9c0e1933.debug...done.
Reading symbols from /usr/lib/debug/.build-id/60/6df9c355103e82140d513bc7a25a635591c153.debug...done.
process 3044 is executing new program: /usr/bin/dpkg
[New process 3044]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[Inferior 3 (process 3044) exited normally]
(gdb) info inferiors
Num Description Executable
1 process 3040 /usr/bin/apt
* 3 <null> /usr/bin/dpkg
(gdb) inferior 1
[Switching to inferior 1 [process 3040] (/usr/bin/apt)]
[Switching to thread 1.1 (Thread 0x7ffff7febb80 (LWP 3040))]
#0 0x00007ffff6f4a38b in __libc_fork () at ../sysdeps/nptl/fork.c:135
135 ../sysdeps/nptl/fork.c: Datei oder Verzeichnis nicht gefunden.
(gdb) cont
Continuing.
[New process 3045]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Reading symbols from /usr/lib/debug/.build-id/77/5143e680ff0cd4cd51cce1ce8ca216e635a1d6.debug...done.
Reading symbols from /usr/lib/debug/.build-id/db/2caeeec37482a98ab1416d0a9afe2944930de9.debug...done.
Reading symbols from /usr/lib/debug/.build-id/ea/d5fd817712e63c1212d1ee7d7ee1b9c29f93a7.debug...done.
Reading symbols from /usr/lib/debug/.build-id/16/d609487bcc4acbac29a4eaa2dda0d2f56211ec.debug...done.
Reading symbols from /usr/lib/debug/.build-id/4e/49714c557ce0472c798f39365ca10f9c0e1933.debug...done.
Reading symbols from /usr/lib/debug/.build-id/60/6df9c355103e82140d513bc7a25a635591c153.debug...done.
process 3045 is executing new program: /usr/bin/dpkg
[New process 3045]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[Inferior 5 (process 3045) exited normally]
(gdb) info inferiors
Num Description Executable
1 process 3040 /usr/bin/apt
3 <null> /usr/bin/dpkg
* 5 <null> /usr/bin/dpkg
(gdb) inferior 1
[Switching to inferior 1 [process 3040] (/usr/bin/apt)]
[Switching to thread 1.1 (Thread 0x7ffff7febb80 (LWP 3040))]
#0 0x00007ffff6f4a38b in __libc_fork () at ../sysdeps/nptl/fork.c:135
135 in ../sysdeps/nptl/fork.c
(gdb) set detach-on-fork on
(gdb) cont
Continuing.
[New process 3046]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
process 3046 is executing new program: /usr/lib/apt/methods/https
[New process 3046]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Thread 7.1 "https" received signal SIGINT, Interrupt.
[Switching to Thread 0x7ffff7fe2400 (LWP 3046)]
0x00007ffff6e989eb in __gconv_load_cache () at gconv_cache.c:62
62 gconv_cache.c: Datei oder Verzeichnis nicht gefunden.
(gdb) info inferiors
Num Description Executable
1 <null> /usr/bin/apt
3 <null> /usr/bin/dpkg
5 <null> /usr/bin/dpkg
6 <null> /usr/bin/apt
* 7 process 3046 /usr/lib/apt/methods/https
(gdb) cont
Continuing.
Thread 7.1 "https" hit Breakpoint 1, main (argv=<optimized out>) at ./methods/https.cc:546
546 ./methods/https.cc: Datei oder Verzeichnis nicht gefunden.
(gdb) bt
#0 main (argv=<optimized out>) at ./methods/https.cc:546
(gdb) stepi
546 in ./methods/https.cc
(gdb)
HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:536
536 in ./methods/https.cc
(gdb) disassemble HttpsMethod::~HttpsMethod
Dump of assembler code for function HttpsMethod::~HttpsMethod():
0x0000555555558f90 <+0>: lea 0x210889(%rip),%rax # 0x555555769820 <_ZTV11HttpsMethod+16>
0x0000555555558f97 <+7>: push %r12
0x0000555555558f99 <+9>: push %rbp
0x0000555555558f9a <+10>: mov %rdi,%rbp
0x0000555555558f9d <+13>: push %rbx
0x0000555555558f9e <+14>: mov %rax,(%rdi)
0x0000555555558fa1 <+17>: mov 0x100(%rdi),%rdi
0x0000555555558fa8 <+24>: callq 0x555555558398
0x0000555555558fad <+29>: lea 0x21099c(%rip),%rax # 0x555555769950 <_ZTV14BaseHttpMethod+16>
0x0000555555558fb4 <+36>: mov 0xd0(%rbp),%rdi
...
0x000055555555916d <+477>: jmpq 0x5555555590e4 <HttpsMethod::~HttpsMethod()+340>
End of assembler dump.
(gdb) stepi
std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose (this=<optimized out>) at /usr/include/c++/6/bits/basic_string.h:180
180 /usr/include/c++/6/bits/basic_string.h: Datei oder Verzeichnis nicht gefunden.
(gdb) display/i $pc
1: x/i $pc
=> 0x55555555873d <main(int, char const**)+173>: add $0x10,%rbp
(gdb) nexti
HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:536
536 ./methods/https.cc: Datei oder Verzeichnis nicht gefunden.
1: x/i $pc
=> 0x555555558741 <main(int, char const**)+177>: mov %rax,0x20(%rsp)
(gdb)
538 in ./methods/https.cc
1: x/i $pc
=> 0x555555558746 <main(int, char const**)+182>: callq 0x555555558398
(gdb) stepi
0x0000555555558398 in ?? ()
1: x/i $pc
=> 0x555555558398: jmpq *0x21193a(%rip) # 0x555555769cd8
(gdb)
curl_easy_cleanup (data=0x55555579f6c0) at easy.c:829
829 easy.c: Datei oder Verzeichnis nicht gefunden.
1: x/i $pc
=> 0x7ffff77d9660 <curl_easy_cleanup>: test %rdi,%rdi
(gdb) bt
#0 curl_easy_cleanup (data=0x55555579f6c0) at easy.c:829
#1 0x000055555555874b in HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:538
#2 main (argv=<optimized out>) at ./methods/https.cc:546
(gdb) disassemble curl_easy_cleanup
Dump of assembler code for function curl_easy_cleanup:
=> 0x00007ffff77d9660 <+0>: test %rdi,%rdi
0x00007ffff77d9663 <+3>: je 0x7ffff77d9670 <curl_easy_cleanup+16>
0x00007ffff77d9665 <+5>: jmpq 0x7ffff77cd4c0 <Curl_close>
0x00007ffff77d966a <+10>: nopw 0x0(%rax,%rax,1)
0x00007ffff77d9670 <+16>: repz retq
End of assembler dump.
(gdb) stepi
0x00007ffff77d9663 829 in easy.c
1: x/i $pc
=> 0x7ffff77d9663 <curl_easy_cleanup+3>: je 0x7ffff77d9670 <curl_easy_cleanup+16>
(gdb)
833 in easy.c
1: x/i $pc
=> 0x7ffff77d9665 <curl_easy_cleanup+5>: jmpq 0x7ffff77cd4c0 <Curl_close>
(gdb)
Curl_close (data=0x55555579f6c0) at url.c:394
394 url.c: Datei oder Verzeichnis nicht gefunden.
1: x/i $pc
=> 0x7ffff77cd4c0 <Curl_close>: test %rdi,%rdi
(gdb) disassemble Curl_close
Dump of assembler code for function Curl_close:
=> 0x00007ffff77cd4c0 <+0>: test %rdi,%rdi
0x00007ffff77cd4c3 <+3>: je 0x7ffff77cd700 <Curl_close+576>
0x00007ffff77cd4c9 <+9>: push %rbp
0x00007ffff77cd4ca <+10>: push %rbx
0x00007ffff77cd4cb <+11>: mov %rdi,%rbx
0x00007ffff77cd4ce <+14>: sub $0x8,%rsp
0x00007ffff77cd4d2 <+18>: callq 0x7ffff77e15c0 <Curl_expire_clear>
0x00007ffff77cd4d7 <+23>: mov 0x60(%rbx),%rdi <<<<<<<<< $rbx contains for OP an invalid pointer
0x00007ffff77cd4db <+27>: test %rdi,%rdi
0x00007ffff77cd4de <+30>: je 0x7ffff77cd4e8 <Curl_close+40>
0x00007ffff77cd4e0 <+32>: mov %rbx,%rsi
...
0x00007ffff77cd702 <+578>: retq
End of assembler dump.
(gdb) b *0x00007ffff77cd4d7
Breakpoint 2 at 0x7ffff77cd4d7: file url.c, line 399.
(gdb) cont
Continuing.
Thread 7.1 "https" hit Breakpoint 2, Curl_close (data=0x55555579f6c0) at url.c:399
399 in url.c
1: x/i $pc
=> 0x7ffff77cd4d7 <Curl_close+23>: mov 0x60(%rbx),%rdi
(gdb) bt
#0 Curl_close (data=0x55555579f6c0) at url.c:399
#1 0x000055555555874b in HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:538
#2 main (argv=<optimized out>) at ./methods/https.cc:546
(gdb) print/x $rip
$1 = 0x7ffff77cd4d7
(gdb) bt full
#0 Curl_close (data=0x55555579f6c0) at url.c:399
m = <optimized out>
#1 0x000055555555874b in HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:538
No locals.
#2 main (argv=<optimized out>) at ./methods/https.cc:546
Binary = "https"
(gdb) down
#1 0x000055555555874b in HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:538
538 ./methods/https.cc: Datei oder Verzeichnis nicht gefunden.
(gdb) down
#0 Curl_close (data=0x55555579f6c0) at url.c:399
399 url.c: Datei oder Verzeichnis nicht gefunden.
(gdb) print data
$15 = (struct Curl_easy *) 0x55555579f6c0
(gdb) print data->multi
$16 = (struct Curl_multi *) 0x0
(gdb) print &(data->multi)
$17 = (struct Curl_multi **) 0x55555579f720
(gdb) print/x 0x55555579f720 - 0x55555579f6c0
$18 = 0x60
(gdb) print/x $rbx
$19 = 0x55555579f6c0
(gdb) up
#1 0x000055555555874b in HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:538
538 ./methods/https.cc: Datei oder Verzeichnis nicht gefunden.
(gdb) print curl
$20 = (CURL *) 0x55555579f6c0
https://sources.debian.org/src/curl/7.52.1-5+deb9u10/lib/url.c/#L399
https://sources.debian.org/src/apt/1.4.10/methods/https.cc/#L538
https://github.com/curl/curl/issues/2764
https://github.com/curl/curl/commit/bc36c4f5463db6ddb098fe83ec7ae8ead012d01b
Reply to: