Bug#783082: linux-image-3.16.0-4-586: video players/browsers crash with 'illegal instruction' on i586

[Bernhard Übelacker <bernhardu@vr-web.de>]

Hello hikaru,
just saw your report and tried if I could reproduce your issue.

But probably you want to reproduce these steps on your hardware to verify
that this is really the issue on real hardware.


These steps I tried to reproduce:
- install a qemu virtual machine with current jessie with some desktop
  environment and following packages: gdb vlc-dbg libav-dbg
- shut down the VM and restarted with forcing it to i586 with these
  parameters: "qemu-system-i386 -cpu pentium ..."
- started vlc without starting a video
- started in a terminal gdb: "gdb --pid 984"
- issue the command "cont"
- play in vlc a video
- at some point I came to the following exception
- the command "bt" shows you the call stack which led to the exception
- (to end the crashed vlc issue commands "kill" and "quit")


Stack without having debug symbols installed:
    Program received signal SIGILL, Illegal instruction.
    [Switching to Thread 0xb17ffb40 (LWP 545)]
    0xaf282c93 in ?? () from /usr/lib/i386-linux-gnu/libavcodec.so.56
    (gdb) bt
    #0  0xaf282c93 in ?? () from /usr/lib/i386-linux-gnu/libavcodec.so.56
    #1  0xaf2b9680 in ?? () from /usr/lib/i386-linux-gnu/libavcodec.so.56
    #2  0xaf2bd816 in ?? () from /usr/lib/i386-linux-gnu/libavcodec.so.56
    #3  0xaf274c86 in ?? () from /usr/lib/i386-linux-gnu/libavcodec.so.56
    #4  0xaf4e9c9c in avcodec_decode_video2 () from /usr/lib/i386-linux-gnu/libavcodec.so.56
    #5  0xb1582801 in ?? () from /usr/lib/vlc/plugins/codec/libavcodec_plugin.so
    #6  0xb743928e in ?? () from /usr/lib/libvlccore.so.8
    #7  0xb743ae57 in ?? () from /usr/lib/libvlccore.so.8
    #8  0xb76a0d97 in start_thread (arg=0xb17ffb40) at pthread_create.c:309
    #9  0xb75f7dfe in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
    (gdb)

With debug symbols installed:
    Program received signal SIGILL, Illegal instruction.
    [Switching to Thread 0xb227fb40 (LWP 1002)]
    0xafcd9c93 in mid_pred (c=0, b=0, a=0) at /build/libav-H9AQHK/libav-11.3/libavcodec/x86/mathops.h:77
    77      /build/libav-H9AQHK/libav-11.3/libavcodec/x86/mathops.h: Datei oder Verzeichnis nicht gefunden.
    (gdb) bt
    #0  0xafcd9c93 in mid_pred (c=0, b=0, a=0) at /build/libav-H9AQHK/libav-11.3/libavcodec/x86/mathops.h:77
    #1  pred_motion (my=<synthetic pointer>, mx=<synthetic pointer>, ref=0, list=0, part_width=4, n=0, h=0xb233e4c0) at /build/libav-H9AQHK/libav-11.3/libavcodec/h264_mvpred.h:140
    #2  ff_h264_decode_mb_cavlc (h=0xb233e4c0) at /build/libav-H9AQHK/libav-11.3/libavcodec/h264_cavlc.c:959
    #3  0xafd10680 in decode_slice (avctx=0xb2338ea0, arg=0xb227f050) at /build/libav-H9AQHK/libav-11.3/libavcodec/h264_slice.c:2275
    #4  0xafd14816 in ff_h264_execute_decode_slices (h=0xb233e4c0, context_count=1) at /build/libav-H9AQHK/libav-11.3/libavcodec/h264_slice.c:2369
    #5  0xafccbc86 in decode_nal_units (parse_extradata=0, buf_size=74, buf=0xb2302500 "", h=0xb233e4c0) at /build/libav-H9AQHK/libav-11.3/libavcodec/h264.c:1662
    #6  h264_decode_frame (avctx=0xb2338ea0, data=0xb2303040, got_frame=0xb227f1a8, avpkt=0xb227f1d4) at /build/libav-H9AQHK/libav-11.3/libavcodec/h264.c:1782
    #7  0xaff40c9c in avcodec_decode_video2 (avctx=0xb2338ea0, picture=0xb2303040, got_picture_ptr=0xb227f1a8, avpkt=0xb227f1d4) at /build/libav-H9AQHK/libav-11.3/libavcodec/utils.c:1600
    #8  0xb2003801 in DecodeVideo (p_dec=0xb2335180, pp_block=0xb227f1a8) at codec/avcodec/video.c:631
    #9  0xb74d428e in DecoderDecodeVideo (p_dec=0xb2335180, p_block=0xb2302490) at input/decoder.c:1385
    #10 0xb74d5e57 in DecoderProcessVideo (b_flush=<optimized out>, p_block=0xb2302490, p_dec=<optimized out>) at input/decoder.c:1631
    #11 DecoderProcess (p_block=<optimized out>, p_dec=<optimized out>) at input/decoder.c:1810
    #12 DecoderThread (p_data=0xb2335180) at input/decoder.c:909
    #13 0xb773bd97 in start_thread (arg=0xb227fb40) at pthread_create.c:309
    #14 0xb7692dfe in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:129
    (gdb)

Current instruction:
   (gdb) display/i $pc
   1: x/i $pc
   => 0xafc39c93 <ff_h264_decode_mb_cavlc+11395>:  cmovg  %edx,%edi


libavcodec/x86/mathops.h:77
    There we find some inline assembly guarded by an #if HAVE_I686.


It looks like that the package libavcodec56 has exactly for
this use case 2 version of the shared object packaged:
    /usr/lib/i386-linux-gnu/libavcodec.so.56.1.0
    /usr/lib/i386-linux-gnu/i686/cmov/libavcodec.so.56.1.0


So I started up a local package build for libav which is still
running with following lines in:

libav-11.3/debian-shared/config.h
  #define LIBAV_CONFIGURATION "--arch=i386 ... --cpu=586"
  #define HAVE_I686 1


So probably this upstream commit could already be sufficient?
  [1] configure: Disable i686 for i586 and lower CPUs


Kind regards,
Bernhard


[1] https://git.libav.org/?p=libav.git;a=commit;h=9030c58a780a02fb8256cb75cd39275eaad786d1