Bug#986832: unblock: svtplay-dl/3.0-2
Package: release.debian.org
Severity: normal
User: release.debian.org@packages.debian.org
Usertags: unblock
Please unblock package svtplay-dl
[ Reason ]
svtplay-dl is a tool to download multimedia from various video websites,
especially targetting nordic demographics. Changes to the external service
svtplay.se required modifications to the svtplay-dl support for that
service to be able to work at all. Because unrelated significant work has
also been done upstream, I've isolated the fix for the service breakage
and backported it in the 3.0-2 package.
[ Impact ]
Users of an unmodified svtplay-dl are not able to use the tool when
interacting with svtplay.se. svtplay.se is the "flagship" service
supported by svtplay-dl. Support for other services is unaffected.
Many of the services covered by svtplay-dl are also covered by other
tools, including youtube-dl. (Though, they also suffer from the same
need for updates as external services change.)
[ Tests ]
Manual testing of svtplay-dl against the svtplay.se service. Changes are
limited to only this service. (The automated test suite of the package
does not catch service breakage.)
[ Risks ]
svtplay-dl is a leaf package and the changes are, while not trivial, at
least limited and scoped to the affected service. The changes are
backported and correspond to the service support in upstream version 3.2.
[ Checklist ]
[x] all changes are documented in the d/changelog
[x] I reviewed all changes and I approve them
[x] attach debdiff against the package in testing
[ Other info ]
The changes are also available on salsa:
https://salsa.debian.org/olof/svtplay-dl/-/compare/debian%2F3.0-1...debian%2F3.0-2
Because of a misunderstanding with the freeze timelines/policies on my
part, the 3.0-2 package is already uploaded to unstable. My apologies if
this causes problems!
Thanks!
--
olof
diffstat for svtplay-dl-3.0 svtplay-dl-3.0
changelog | 6
patches/debian-changes | 302 +++++++++++++++++++++++++++++++++++++++++++++++++
patches/series | 1
3 files changed, 309 insertions(+)
No differences were encountered between the control files
diff -Nru svtplay-dl-3.0/debian/changelog svtplay-dl-3.0/debian/changelog
--- svtplay-dl-3.0/debian/changelog 2021-02-17 22:21:05.000000000 +0100
+++ svtplay-dl-3.0/debian/changelog 2021-03-12 19:21:49.000000000 +0100
@@ -1,3 +1,9 @@
+svtplay-dl (3.0-2) unstable; urgency=medium
+
+ * Backport fixes to svtplay service support from 3.2
+
+ -- Olof Johansson <olof@ethup.se> Fri, 12 Mar 2021 19:21:49 +0100
+
svtplay-dl (3.0-1) unstable; urgency=medium
* New upstream version 3.0
diff -Nru svtplay-dl-3.0/debian/patches/debian-changes svtplay-dl-3.0/debian/patches/debian-changes
--- svtplay-dl-3.0/debian/patches/debian-changes 1970-01-01 01:00:00.000000000 +0100
+++ svtplay-dl-3.0/debian/patches/debian-changes 2021-03-12 19:21:49.000000000 +0100
@@ -0,0 +1,302 @@
+Subject: Collected Debian patches for svtplay-dl
+Author: Olof Johansson <olof@ethup.se>
+
+The svtplay-dl package is maintained in Git rather than maintaining
+patches as separate files, and separating the patches doesn't seem to
+be worth the effort. They are therefore all included in this single
+Debian patch.
+
+For full commit history and separated commits, see the packaging Git
+repository.
+--- svtplay-dl-3.0.orig/lib/svtplay_dl/service/svtplay.py
++++ svtplay-dl-3.0/lib/svtplay_dl/service/svtplay.py
+@@ -7,7 +7,6 @@ import json
+ import logging
+ import re
+ import time
+-from operator import itemgetter
+ from urllib.parse import parse_qs
+ from urllib.parse import urljoin
+ from urllib.parse import urlparse
+@@ -21,6 +20,13 @@ from svtplay_dl.subtitle import subtitle
+ from svtplay_dl.utils.text import filenamify
+
+ URL_VIDEO_API = "https://api.svt.se/video/";
++LIVE_CHANNELS = {
++ "svtbarn": "ch-barnkanalen",
++ "svt1": "ch-svt1",
++ "svt2": "ch-svt2",
++ "svt24": "ch-svt24",
++ "kunskapskanalen": "ch-kunskapskanalen",
++}
+
+
+ class Svtplay(Service, MetadataThumbMixin):
+@@ -42,7 +48,7 @@ class Svtplay(Service, MetadataThumbMixi
+ urldata = self.get_urldata()
+
+ if parse.path[:8] == "/kanaler":
+- ch = "ch-{}".format(parse.path[parse.path.rfind("/") + 1 :])
++ ch = LIVE_CHANNELS[parse.path[parse.path.rfind("/") + 1 :]]
+ _url = urljoin(URL_VIDEO_API, ch)
+ res = self.http.get(_url)
+ try:
+@@ -66,14 +72,12 @@ class Svtplay(Service, MetadataThumbMixi
+ for data_entry in janson["props"]["urqlState"].values():
+ entry = json.loads(data_entry["data"])
+ for key, data in entry.items():
+- if key == "listablesByEscenicId" and "videoSvtId" in data[0]:
+- video_data = data[0]
+- vid = video_data["videoSvtId"]
++ if key == "detailsPage" and "moreDetails" in data:
++ video_data = data
++ vid = data["video"]["svtId"]
+ break
+- # if video_data:
+- # break
+
+- if not vid and not self.visibleid:
++ if not vid:
+ yield ServiceError("Can't find video id")
+ return
+
+@@ -107,7 +111,6 @@ class Svtplay(Service, MetadataThumbMixi
+ query = parse_qs(urlparse(i["url"]).query)
+ if "alt" in query and len(query["alt"]) > 0:
+ alt = self.http.get(query["alt"][0])
+-
+ if i["format"][:3] == "hls":
+ streams = hlsparse(self.config, self.http.request("get", i["url"]), i["url"], output=self.output)
+ if alt:
+@@ -124,56 +127,23 @@ class Svtplay(Service, MetadataThumbMixi
+ for n in list(alt_streams.keys()):
+ yield alt_streams[n]
+
+- def _get_visibleid(self, janson):
+- esceni = None
++ def _last_chance(self):
++ videos = []
++ match = re.search(self.info_search_expr, self.get_urldata())
++ if not match:
++ logging.error("Can't find video info.")
++ return
++ janson = json.loads(match.group(1))
++ video_data = None
+ for data_entry in janson["props"]["urqlState"].values():
+ entry = json.loads(data_entry["data"])
+- for key in entry.keys():
+- if "listablesBy" in key:
+- esceni = entry[key]
+- break
+- if esceni:
+- break
+-
+- if esceni:
+- try:
+- return esceni[0]["id"]
+- except IndexError:
+- return None
+- else:
+- return esceni
+-
+- def _last_chance(self, videos, page, maxpage=2):
+- if page > maxpage:
+- return videos
+-
+- res = self.http.get("http://www.svtplay.se/sista-chansen?sida={}".format(page))
+- match = re.search("__svtplay'] = ({.*});", res.text)
+- if not match:
++ for key, data in entry.items():
++ if key == "startForSvtPlay":
++ video_data = data
++ if not video_data:
+ return videos
+-
+- dataj = json.loads(match.group(1))
+- pages = dataj["gridPage"]["pagination"]["totalPages"]
+-
+- for i in dataj["gridPage"]["content"]:
+- videos.append(i["contentUrl"])
+- page += 1
+- videos.extend(self._last_chance(videos, page, pages))
+- return videos
+-
+- def _genre(self, jansson):
+- videos = []
+- parse = urlparse(self._url)
+- dataj = jansson["clusterPage"]
+- tab = re.search("tab=(.+)", parse.query)
+- if tab:
+- tab = tab.group(1)
+- for i in dataj["tabs"]:
+- if i["slug"] == tab:
+- videos = self.videos_to_list(i["content"], videos)
+- else:
+- videos = self.videos_to_list(dataj["clips"], videos)
+-
++ for i in video_data["selections"][0]["items"]:
++ videos.append(i["item"]["urls"]["svtplay"])
+ return videos
+
+ def find_all_episodes(self, config):
+@@ -187,7 +157,7 @@ class Svtplay(Service, MetadataThumbMixi
+ tab = query["tab"][0]
+
+ if re.search("sista-chansen", parse.path):
+- videos = self._last_chance(videos, 1)
++ videos = self._last_chance()
+ else:
+ match = re.search(self.info_search_expr, self.get_urldata())
+ if not match:
+@@ -195,30 +165,17 @@ class Svtplay(Service, MetadataThumbMixi
+ return
+
+ janson = json.loads(match.group(1))
+- self.visibleid = self._get_visibleid(janson)
+- if not self.visibleid:
+- logging.error("Can't find video id. removed?")
+- return
+-
+- match = re.search(self.info_search_expr, self.get_urldata())
+- if not match:
+- logging.error("Can't find video info.")
+- return videos
+- janson = json.loads(match.group(1))
+- associatedContent = None
+-
+- for json_entry in janson["props"]["urqlState"].values():
+- entry = json.loads(json_entry["data"])
++ video_data = None
++ for data_entry in janson["props"]["urqlState"].values():
++ entry = json.loads(data_entry["data"])
+ for key, data in entry.items():
+- if "listablesBy" in key and data[0]["associatedContent"][0]["id"] != "related":
+- associatedContent = data[0]["associatedContent"]
++ if key == "detailsPage":
++ video_data = data
+ break
+- if associatedContent:
+- break
+
+ collections = []
+ videos = []
+- for i in associatedContent:
++ for i in video_data["associatedContent"]:
+ if tab:
+ if tab == i["id"]:
+ collections.append(i)
+@@ -235,41 +192,21 @@ class Svtplay(Service, MetadataThumbMixi
+
+ for i in collections:
+ for epi in i["items"]:
+- if "variants" in epi["item"]:
+- for variant in epi["item"]["variants"]:
+- if variant["urls"]["svtplay"] not in videos:
+- videos.append(variant["urls"]["svtplay"])
+ if epi["item"]["urls"]["svtplay"] not in videos:
+ videos.append(epi["item"]["urls"]["svtplay"])
+
+ episodes = [urljoin("http://www.svtplay.se";, x) for x in videos]
+-
+ if config.get("all_last") > 0:
+ return episodes[-config.get("all_last") :]
+ return episodes
+
+- def videos_to_list(self, lvideos, videos):
+- if "episodeNumber" in lvideos[0] and lvideos[0]["episodeNumber"]:
+- lvideos = sorted(lvideos, key=itemgetter("episodeNumber"))
+- for n in lvideos:
+- parse = urlparse(n["contentUrl"])
+- if parse.path not in videos:
+- videos.append(parse.path)
+- if "versions" in n:
+- for i in n["versions"]:
+- parse = urlparse(i["contentUrl"])
+- if parse.path not in videos:
+- videos.append(parse.path)
+-
+- return videos
+-
+ def outputfilename(self, data):
+ name = None
+ desc = None
+
+- name = data["parent"]["slug"]
+- other = data["slug"]
+- vid = data["id"]
++ name = data["moreDetails"]["heading"]
++ other = data["moreDetails"]["episodeHeading"]
++ vid = data["video"]["svtId"]
+ id = hashlib.sha256(vid.encode("utf-8")).hexdigest()[:7]
+
+ if name == other:
+@@ -299,13 +236,7 @@ class Svtplay(Service, MetadataThumbMixi
+ def seasoninfo(self, data):
+ season, episode = None, None
+
+- if "episode" not in data:
+- return season, episode
+-
+- if "positionInSeason" not in data["episode"]:
+- return season, episode
+-
+- match = re.search(r"Säsong (\d+) — Avsnitt (\d+)", data["episode"]["positionInSeason"])
++ match = re.search(r"/säsong (\d+)/avsnitt (\d+)", data["analyticsIdentifiers"]["viewId"])
+ if not match:
+ return season, episode
+
+@@ -324,27 +255,27 @@ class Svtplay(Service, MetadataThumbMixi
+ value = value[:-3] + value[-2:]
+ return value
+
+- validfrom = episode["validFrom"]
++ validfrom = episode["item"]["validFrom"]
+ if "+" in validfrom:
+ date = time.mktime(
+ datetime.datetime.strptime(
+- _fix_broken_timezone_implementation(episode["validFrom"].replace("Z", "")),
++ _fix_broken_timezone_implementation(episode["item"]["validFrom"].replace("Z", "")),
+ "%Y-%m-%dT%H:%M:%S%z",
+ ).timetuple(),
+ )
+ else:
+ date = time.mktime(
+ datetime.datetime.strptime(
+- _fix_broken_timezone_implementation(episode["validFrom"].replace("Z", "")),
++ _fix_broken_timezone_implementation(episode["item"]["validFrom"].replace("Z", "")),
+ "%Y-%m-%dT%H:%M:%S",
+ ).timetuple(),
+ )
+ self.output["publishing_datetime"] = int(date)
+
+- self.output["title_nice"] = episode["parent"]["name"]
++ self.output["title_nice"] = episode["moreDetails"]["heading"]
+
+ try:
+- t = episode["parent"]["image"]
++ t = episode["item"]["parent"]["image"]["wide"]
+ except KeyError:
+ t = ""
+ if isinstance(t, dict):
+@@ -355,7 +286,7 @@ class Svtplay(Service, MetadataThumbMixi
+ url = t.format(format="large")
+ self.output["showthumbnailurl"] = url
+ try:
+- t = episode["image"]
++ t = episode["images"]["wide"]
+ except KeyError:
+ t = ""
+ if isinstance(t, dict):
+@@ -366,8 +297,8 @@ class Svtplay(Service, MetadataThumbMixi
+ url = t.format(format="large")
+ self.output["episodethumbnailurl"] = url
+
+- if "longDescription" in episode["parent"]:
+- self.output["showdescription"] = episode["parent"]["longDescription"]
++ if "longDescription" in episode["item"]["parent"]:
++ self.output["showdescription"] = episode["item"]["parent"]["longDescription"]
+
+- if "longDescription" in episode:
+- self.output["episodedescription"] = episode["longDescription"]
++ if "description" in episode:
++ self.output["episodedescription"] = episode["description"]
diff -Nru svtplay-dl-3.0/debian/patches/series svtplay-dl-3.0/debian/patches/series
--- svtplay-dl-3.0/debian/patches/series 1970-01-01 01:00:00.000000000 +0100
+++ svtplay-dl-3.0/debian/patches/series 2021-03-12 19:21:49.000000000 +0100
@@ -0,0 +1 @@
+debian-changes
Reply to: