Bug#973942: nemo: thumbnailing doesn't work for all kinds of videos

[Christoph Anton Mitterer <calestyo@scientia.net>]

Hey there.

I finally found some time to look again into this, and what was
previously one issue, seem to be now even more ;-)




1) First, sharing the videos which failed back then (and still do) is a
bit problematic, cause they contain private content like peoples faces
from a party and so on.

What happens with those videos is basically:
When I
  rm -rf ~/.cache/thumbnails/fail/
  (do this again after every test, otherwise nemo will ignore the
  already failed files)
nemo will show me icons with a clock for these videos (that haven't a
thumbnail yet), kinda indicating that it's still building up.

at the same time I get one (so it doesn't do it concurrently for
multiple videos) process:
  /usr/bin/totem-video-thumbnailer -s 256 file:///home/calestyo/v/someVideo.mp4 /tmp/.gnome_desktop_thumbnail.1V6120

This hangs there for a *long* time... at least several minutes, then
I've killed it (maybe it hangs forever it doesn't seem to be just the
30s mentioned in totem-video-thumbnailer --help).

So next to the problem that it doesn't work at all, it's not killed
fast enough IMO... guess something like a few seconds should be enough?


Having started nemo with debugging like:
nemo -q ; reset ; NEMO_DEBUG=thumbnails nemo --debug .

shows me amongst other messages:
(nemo:70867): Gtk-WARNING **: 17:10:26.209: GtkLabel 0x55c9ae8f36a0 adjusted size horizontal min 17 natural 59 must not decrease below min 19 natural 61

(nemo:70867): Gtk-WARNING **: 17:10:26.211: GtkLabel 0x55c9ae8f36a0 attempted to adjust its size allocation from 200,5 61x28 to 199,5 63x28. adjust_size_allocation must keep allocation inside original bounds
(nemo:70867): dconf-DEBUG: 17:10:26.340: watch_fast: "/org/cinnamon/desktop/thumbnailers/" (establishing: 0, active: 0)
(nemo:70867): dconf-DEBUG: 17:10:26.341: watch_established: "/org/cinnamon/desktop/thumbnailers/" (establishing: 1)
(totem-video-thumbnailer:70881): GLib-GIO-DEBUG: 17:10:26.450: _g_io_module_get_default: Found default implementation gvfs (GDaemonVfs) for ‘gio-vfs’
(totem-video-thumbnailer:70881): GStreamer-WARNING **: 17:10:26.701: failed to create thread: Error creating thread: Resource temporarily unavailable
(totem-video-thumbnailer:70881): GStreamer-WARNING **: 17:10:26.704: failed to create thread: Error creating thread: Resource temporarily unavailable
** (totem-video-thumbnailer:70881): WARNING **: 17:10:26.704: Could not take screenshot: failed to retrieve or convert video frame
** (totem-video-thumbnailer:70881): WARNING **: 17:10:26.704: Could not take screenshot: failed to retrieve or convert video frame
** (totem-video-thumbnailer:70881): WARNING **: 17:10:26.704: Could not take screenshot: failed to retrieve or convert video frame
** (totem-video-thumbnailer:70881): WARNING **: 17:10:26.705: Could not take screenshot: failed to retrieve or convert video frame
** (totem-video-thumbnailer:70881): WARNING **: 17:10:26.705: Could not take screenshot: failed to retrieve or convert video frame
(nemo:70867): Gtk-WARNING **: 17:10:31.099: GtkLabel 0x55c9ae8f36a0 adjusted size horizontal min 17 natural 59 must not decrease below min 19 natural 61
(nemo:70867): Gtk-WARNING **: 17:10:31.104: GtkLabel 0x55c9ae8f36a0 attempted to adjust its size allocation from 200,5 61x28 to 199,5 63x28. adjust_size_allocation must keep allocation inside original bounds

(I removed empty lines from the log)


Running the same command manually with verbose:
$ /usr/bin/totem-video-thumbnailer --verbose -s 256 file:///home/calestyo/v/someVideo.mp4 /tmp/.gnome_desktop_thumbnail.G9Q220
TotemVideoThumbnailer-Message: 17:23:12.138: Initialised libraries, about to create video widget
TotemVideoThumbnailer-Message: 17:23:12.144: setting URI file:///home/calestyo/v/someVideo.mp4
TotemVideoThumbnailer-Message: 17:23:12.144: Video widget created
TotemVideoThumbnailer-Message: 17:23:12.144: About to open video file
(totem-video-thumbnailer:71645): GStreamer-WARNING **: 17:23:12.334: failed to create thread: Error creating thread: Resource temporarily unavailable
(totem-video-thumbnailer:71645): GStreamer-WARNING **: 17:23:12.336: failed to create thread: Error creating thread: Resource temporarily unavailable
TotemVideoThumbnailer-Message: 17:23:12.337: Checking whether file has cover
TotemVideoThumbnailer-Message: 17:23:12.338: Opened video file: 'file:///home/calestyo/v/someVideo.mp4'
TotemVideoThumbnailer-Message: 17:23:12.338: About to seek to 0,333333
TotemVideoThumbnailer-Message: 17:23:12.338: About to get frame for iter 0
** (totem-video-thumbnailer:71645): WARNING **: 17:23:12.338: Could not take screenshot: failed to retrieve or convert video frame
TotemVideoThumbnailer-Message: 17:23:12.339: Frame for iter 0 was not interesting
TotemVideoThumbnailer-Message: 17:23:12.339: About to seek to 0,666667
TotemVideoThumbnailer-Message: 17:23:12.339: About to get frame for iter 1
** (totem-video-thumbnailer:71645): WARNING **: 17:23:12.339: Could not take screenshot: failed to retrieve or convert video frame
TotemVideoThumbnailer-Message: 17:23:12.339: Frame for iter 1 was not interesting
TotemVideoThumbnailer-Message: 17:23:12.339: About to seek to 0,100000
TotemVideoThumbnailer-Message: 17:23:12.339: About to get frame for iter 2
** (totem-video-thumbnailer:71645): WARNING **: 17:23:12.339: Could not take screenshot: failed to retrieve or convert video frame
TotemVideoThumbnailer-Message: 17:23:12.339: Frame for iter 2 was not interesting
TotemVideoThumbnailer-Message: 17:23:12.339: About to seek to 0,900000
TotemVideoThumbnailer-Message: 17:23:12.339: About to get frame for iter 3
** (totem-video-thumbnailer:71645): WARNING **: 17:23:12.339: Could not take screenshot: failed to retrieve or convert video frame
TotemVideoThumbnailer-Message: 17:23:12.339: Frame for iter 3 was not interesting
TotemVideoThumbnailer-Message: 17:23:12.339: About to seek to 0,500000
TotemVideoThumbnailer-Message: 17:23:12.339: About to get frame for iter 4
** (totem-video-thumbnailer:71645): WARNING **: 17:23:12.339: Could not take screenshot: failed to retrieve or convert video frame
TotemVideoThumbnailer-Message: 17:23:12.339: Frame for iter 4 was not interesting

(again, empty lines removed)


With gstreamer debug level 3:
$ /usr/bin/totem-video-thumbnailer --verbose --gst-debug-level=3 -s 256 file:///home/calestyo/v/someVideo.mp4 /tmp/.gnome_desktop_thumbnail.G9Q220
TotemVideoThumbnailer-Message: 17:31:03.048: Initialised libraries, about to create video widget
TotemVideoThumbnailer-Message: 17:31:03.054: setting URI file:///home/calestyo/v/someVideo.mp4
TotemVideoThumbnailer-Message: 17:31:03.054: Video widget created
TotemVideoThumbnailer-Message: 17:31:03.054: About to open video file
0:00:00.019466043 72063 0x55846d73b950 WARN                 basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<source> pad not activated yet
0:00:00.019739829 72063 0x55846d73b950 WARN                 basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<source> pad not activated yet
0:00:00.033081958 72063 0x55846d759760 WARN                 qtdemux qtdemux_types.c:244:qtdemux_type_get: unknown QuickTime node type pasp
0:00:00.033105243 72063 0x55846d759760 WARN                 qtdemux qtdemux_types.c:244:qtdemux_type_get: unknown QuickTime node type sgpd
0:00:00.033111491 72063 0x55846d759760 WARN                 qtdemux qtdemux_types.c:244:qtdemux_type_get: unknown QuickTime node type sbgp
0:00:00.033149992 72063 0x55846d759760 WARN                 qtdemux qtdemux.c:3066:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 1
0:00:00.033319557 72063 0x55846d759760 WARN                 qtdemux qtdemux.c:3066:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 2
(totem-video-thumbnailer:72063): GStreamer-WARNING **: 17:31:03.251: failed to create thread: Error creating thread: Resource temporarily unavailable
0:00:00.216663276 72063 0x7f2b7c1eb6a0 WARN                GST_PADS gstpad.c:1142:gst_pad_set_active:<vqueue:src> Failed to activate pad
0:00:00.218351100 72063 0x7f2b7c1eb6a0 WARN                playsink gstplaysink.c:2893:gen_audio_chain:<playsink> warning: No volume control found
0:00:00.218373365 72063 0x7f2b7c1eb6a0 WARN                playsink gstplaysink.c:2893:gen_audio_chain:<playsink> warning: Volume/mute is not available
(totem-video-thumbnailer:72063): GStreamer-WARNING **: 17:31:03.254: failed to create thread: Error creating thread: Resource temporarily unavailable
0:00:00.218706244 72063 0x7f2b7c1eb6a0 WARN                GST_PADS gstpad.c:1142:gst_pad_set_active:<aqueue:src> Failed to activate pad
TotemVideoThumbnailer-Message: 17:31:03.254: Checking whether file has cover
0:00:00.219015932 72063 0x7f2b7c1eb6a0 WARN                GST_PADS gstpad.c:4303:gst_pad_peer_query:<streamsynchronizer0:src_1> could not send sticky events
TotemVideoThumbnailer-Message: 17:31:03.254: Opened video file: 'file:///home/calestyo/v/someVideo.mp4'
0:00:00.219088690 72063 0x7f2b7c1eb6a0 WARN                GST_PADS gstpad.c:4303:gst_pad_peer_query:<streamsynchronizer0:src_1> could not send sticky events
TotemVideoThumbnailer-Message: 17:31:03.254: About to seek to 0,333333
TotemVideoThumbnailer-Message: 17:31:03.254: About to get frame for iter 0
0:00:00.219102191 72063 0x7f2b7c1eb4c0 WARN                GST_PADS gstpad.c:4303:gst_pad_peer_query:<streamsynchronizer0:src_0> could not send sticky events
** (totem-video-thumbnailer:72063): WARNING **: 17:31:03.254: Could not take screenshot: failed to retrieve or convert video frame
0:00:00.219373779 72063 0x7f2b7c1eb4c0 WARN                GST_PADS gstpad.c:4303:gst_pad_peer_query:<streamsynchronizer0:src_0> could not send sticky events
TotemVideoThumbnailer-Message: 17:31:03.255: Frame for iter 0 was not interesting
TotemVideoThumbnailer-Message: 17:31:03.255: About to seek to 0,666667
TotemVideoThumbnailer-Message: 17:31:03.255: About to get frame for iter 1
** (totem-video-thumbnailer:72063): WARNING **: 17:31:03.255: Could not take screenshot: failed to retrieve or convert video frame
TotemVideoThumbnailer-Message: 17:31:03.255: Frame for iter 1 was not interesting
TotemVideoThumbnailer-Message: 17:31:03.255: About to seek to 0,100000
TotemVideoThumbnailer-Message: 17:31:03.255: About to get frame for iter 2
** (totem-video-thumbnailer:72063): WARNING **: 17:31:03.255: Could not take screenshot: failed to retrieve or convert video frame
TotemVideoThumbnailer-Message: 17:31:03.255: Frame for iter 2 was not interesting
TotemVideoThumbnailer-Message: 17:31:03.255: About to seek to 0,900000
TotemVideoThumbnailer-Message: 17:31:03.255: About to get frame for iter 3
** (totem-video-thumbnailer:72063): WARNING **: 17:31:03.255: Could not take screenshot: failed to retrieve or convert video frame
TotemVideoThumbnailer-Message: 17:31:03.255: Frame for iter 3 was not interesting
TotemVideoThumbnailer-Message: 17:31:03.255: About to seek to 0,500000
TotemVideoThumbnailer-Message: 17:31:03.255: About to get frame for iter 4
** (totem-video-thumbnailer:72063): WARNING **: 17:31:03.255: Could not take screenshot: failed to retrieve or convert video frame
TotemVideoThumbnailer-Message: 17:31:03.255: Frame for iter 4 was not interesting

(with level 4 it gets already pretty big, but I can send it if it
helps)




2) I tried to look for some video with which you can reproduce, e.g.
https://www.youtube.com/watch?v=sC9abcLLQpI

$ ffprobe Rogue\ One\ -\ A\ Star\ Wars\ Story\ Trailer\ #2\ \(Official\)-sC9abcLLQpI.mp4 2>&1 | grep Stream
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 1171 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 47.95 tbc (default)
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
$ ffprobe ~/v/someVideo.mp4 2>&1 | grep Stream
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 712 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 139 kb/s (default)

Pretty similar I'd say?!


Running nemo with debug there:
** (nemo:72313): WARNING **: 17:33:36.445: The theme appears to have no nemo support.  Adding some...
** (nemo:72313): DEBUG: 17:33:36.448: Open called on the GApplication instance; 1 files
(nemo:72313): XApp-DEBUG: 17:33:36.562: XAppFavorites: init:
(nemo:72313): dconf-DEBUG: 17:33:36.562: watch_fast: "/org/x/apps/favorites/" (establishing: 0, active: 0)
(nemo:72313): XApp-DEBUG: 17:33:36.562: XAppFavorites: load_favorite: favorites loaded (0)
(nemo:72313): dconf-DEBUG: 17:33:36.563: watch_established: "/org/x/apps/favorites/" (establishing: 1)
(nemo:72313): XApp-DEBUG: 17:33:36.563: XAppFavorites: get_n_favorites returning number of items: 0.
(nemo:72313): dconf-DEBUG: 17:33:36.568: change_fast
(nemo:72313): dconf-DEBUG: 17:33:36.568: change_notify: /org/nemo/window-state/start-with-sidebar
(nemo:72313): XApp-DEBUG: 17:33:36.580: XAppFavorites: get_n_favorites returning number of items: 0.
(nemo:72313): dconf-DEBUG: 17:33:36.641: change_fast
(nemo:72313): dconf-DEBUG: 17:33:36.641: change_notify: /org/nemo/window-state/maximized
(nemo:72313): XApp-DEBUG: 17:33:36.642: XAppFavorites: get_n_favorites returning number of items: 0.
(nemo:72313): dconf-DEBUG: 17:33:36.659: change_fast
(nemo:72313): dconf-DEBUG: 17:33:36.659: change_notify: /org/nemo/window-state/maximized
(nemo:72313): dconf-DEBUG: 17:33:36.673: change_fast
(nemo:72313): dconf-DEBUG: 17:33:36.673: change_notify: /org/nemo/window-state/maximized
(nemo:72313): dconf-DEBUG: 17:33:36.850: watch_fast: "/org/cinnamon/desktop/thumbnailers/" (establishing: 0, active: 0)
(nemo:72313): dconf-DEBUG: 17:33:36.853: watch_established: "/org/cinnamon/desktop/thumbnailers/" (establishing: 1)
(totem-video-thumbnailer:72326): GLib-GIO-DEBUG: 17:33:36.914: _g_io_module_get_default: Found default implementation gvfs (GDaemonVfs) for ‘gio-vfs’
(totem-video-thumbnailer:72326): GLib-ERROR **: 17:33:37.090: ../../../glib/gmem.c:112: failed to allocate 3145167 bytes
(nemo:72313): CinnamonDesktop-WARNING **: 17:33:37.093: Unable to create loader for mime type video/mp4: Unrecognized image file format
(nemo:72313): CinnamonDesktop-WARNING **: 17:33:37.094: Error creating thumbnail for file:///home/calestyo/v/v2/Rogue%20One%20-%20A%20Star%20Wars%20Story%20Trailer%20%232%20(Official)-sC9abcLLQpI.mp4: Unrecognized image file format
^C

Doesn't seem to even start the thumbnailer process (or it dies of so
quickly that I don't see it).

Strangely enough, video/mp4 is listed in
/usr/share/thumbnailers/totem.thumbnailer



With https://www.youtube.com/watch?v=ZSzeFFsKEt4 I get a similar:
(nemo:75710): dconf-DEBUG: 17:48:32.988: watch_established: "/org/cinnamon/desktop/thumbnailers/" (establishing: 1)
(totem-video-thumbnailer:75719): GLib-GIO-DEBUG: 17:48:33.032: _g_io_module_get_default: Found default implementation gvfs (GDaemonVfs) for ‘gio-vfs’
***MEMORY-ERROR***: totem-video-thumbnailer[75719]: GSlice: failed to allocate 4080 bytes (alignment: 4096): Cannot allocate memory
(nemo:75710): CinnamonDesktop-WARNING **: 17:48:33.237: Unable to create loader for mime type video/mp4: Unrecognized image file format
(nemo:75710): CinnamonDesktop-WARNING **: 17:48:33.237: Error creating thumbnail for file:///home/calestyo/v/v3/x.mp4: Unrecognized image file format
^C

With a nice MEMORY-ERROR.


With https://file-examples-com.github.io/uploads/2017/04/file_example_MP4_1920_18MG.mp4 :

nemo with debug:
(nemo:76023): dconf-DEBUG: 17:53:17.588: watch_established: "/org/cinnamon/desktop/thumbnailers/" (establishing: 1)
(totem-video-thumbnailer:76032): GLib-GIO-DEBUG: 17:53:17.638: _g_io_module_get_default: Found default implementation gvfs (GDaemonVfs) for ‘gio-vfs’
(totem-video-thumbnailer:76032): GStreamer-WARNING **: 17:53:17.806: failed to create thread: Error creating thread: Resource temporarily unavailable
(totem-video-thumbnailer:76032): GLib-ERROR **: 17:53:17.815: ../../../glib/gmem.c:112: failed to allocate 3145167 bytes
(nemo:76023): CinnamonDesktop-WARNING **: 17:53:17.819: Unable to create loader for mime type video/mp4: Unrecognized image file format
(nemo:76023): CinnamonDesktop-WARNING **: 17:53:17.820: Error creating thumbnail for file:///home/calestyo/v/v3/file_example_MP4_1920_18MG.mp4: Unrecognized image file format
(nemo:76023): dconf-DEBUG: 17:53:22.076: change_fast
(nemo:76023): dconf-DEBUG: 17:53:22.077: change_notify: /org/nemo/window-state/maximized

and a nice:
May 03 17:53:17 heisenberg kernel: traps: multiqueue0:src[76038] trap int3 ip:7f8d90d49ca7 sp:7f8d85ff9a70 error:0 in libglib-2.0.so.0.6600.8[7f8d90d0d000+88000]


Running the thumbnailer on it directly:
$ /usr/bin/totem-video-thumbnailer --verbose --gst-debug-level=3 -s 256 file:///home/calestyo/v/v3/file_example_MP4_1920_18MG.mp4 foo
TotemVideoThumbnailer-Message: 17:54:13.018: Initialised libraries, about to create video widget
TotemVideoThumbnailer-Message: 17:54:13.025: setting URI file:///home/calestyo/v/v3/file_example_MP4_1920_18MG.mp4
TotemVideoThumbnailer-Message: 17:54:13.025: Video widget created
TotemVideoThumbnailer-Message: 17:54:13.025: About to open video file
0:00:00.035308643 76083 0x55b5cdc2a350 WARN                 basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<source> pad not activated yet
0:00:00.035646818 76083 0x55b5cdc2a350 WARN                 basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<source> pad not activated yet
0:00:00.051052449 76083 0x55b5cdc45b60 WARN                 qtdemux qtdemux_types.c:244:qtdemux_type_get: unknown QuickTime node type iods
0:00:00.051073944 76083 0x55b5cdc45b60 WARN                 qtdemux qtdemux_types.c:244:qtdemux_type_get: unknown QuickTime node type pasp
0:00:00.051081091 76083 0x55b5cdc45b60 WARN                 qtdemux qtdemux_types.c:244:qtdemux_type_get: unknown QuickTime node type sgpd
0:00:00.051085850 76083 0x55b5cdc45b60 WARN                 qtdemux qtdemux_types.c:244:qtdemux_type_get: unknown QuickTime node type sbgp
0:00:00.051134666 76083 0x55b5cdc45b60 WARN                 qtdemux qtdemux.c:3066:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 1
0:00:00.051203973 76083 0x55b5cdc45b60 WARN                 qtdemux qtdemux.c:3066:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 2
(totem-video-thumbnailer:76083): GStreamer-WARNING **: 17:54:13.178: failed to create thread: Error creating thread: Resource temporarily unavailable
(totem-video-thumbnailer:76083): GLib-ERROR **: 17:54:13.188: ../../../glib/gmem.c:112: failed to allocate 3145167 bytes



3) Last but not least I've removed totem again, and installed
ffmpegthumbnailer, wich also provides a file in:
/usr/share/thumbnailers/

Strangely enough, nemo doesn't seem to try anything at all here - no
errors, no thumbnailing either.

Any ideas why?


It would be nice if there was a virtual package like freedesktop-
thumbnailer or so, which all packages that have a compatible
thumbnailer provide.
nemo/nautilus/etc. could than recommend one of them which is kinda
general-purpose with the virtual package as alternative (plus telling
in the package description, one must select all those which one wants),
like
Recommends: totem | freedesktop-thumbnailer

Plus it would be nice if the totem thumbnailer is split in a separate
package.


Thanks,
Chris