Your message dated Fri, 10 May 2019 20:49:12 +0200 with message-id <19aa2195-a203-176d-644d-ecc6506d86fd@debian.org> and subject line Re: unblock: jupyter-notebook/5.7.8-1 has caused the Debian Bug report #926594, regarding unblock: jupyter-notebook/5.7.8-1 to be marked as done. This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact owner@bugs.debian.org immediately.) -- 926594: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926594 Debian Bug Tracking System Contact owner@bugs.debian.org with problems
--- Begin Message ---
- To: Debian Bug Tracking System <submit@bugs.debian.org>
- Subject: unblock: jupyter-notebook/5.7.8-1
- From: Gordon Ball <gordon@chronitis.net>
- Date: Sun, 07 Apr 2019 13:38:29 +0000
- Message-id: <155464430972.4530.14551433539084466581.reportbug@emissary.chronitis.net>
Package: release.debian.org Severity: normal User: release.debian.org@packages.debian.org Usertags: unblock Please unblock package jupyter-notebook, 5.7.4-2.1 -> 5.7.8-1 (pending approval before the latter version is uploaded to unstable). There are two new CVEs since 5.7.4: * CVE-2019-9644 (#924515) * CVE-2019-10255 (#925939) The diff between 5.7.4 and 5.7.8 upstream consists mostly of fixes for these issues. There are also a couple of small non-security related bug fixes. In principle two of these fixes are not needed (one concerning MIME types relevant only on Windows, one concerning compatibility with a newer major version of tornado, which is not yet in debian), but it seems preferable to use the upstream changes unmodified rather than selectively remove a small fraction of them. unblock jupyter-notebook/5.7.8-1 -- System Information: Debian Release: buster/sid APT prefers unstable APT policy: (500, 'unstable') Architecture: amd64 (x86_64) Kernel: Linux 4.19.0-3-amd64 (SMP w/1 CPU core) Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE=en_GB (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enableddiff -Nru jupyter-notebook-5.7.4/debian/changelog jupyter-notebook-5.7.8/debian/changelog --- jupyter-notebook-5.7.4/debian/changelog 2019-03-30 14:52:25.000000000 +0000 +++ jupyter-notebook-5.7.8/debian/changelog 2019-04-07 11:46:04.000000000 +0000 @@ -1,3 +1,11 @@ +jupyter-notebook (5.7.8-1) unstable; urgency=medium + + * New upstream release 5.7.8 + * Fixes CVE-2019-9644 (Closes: #924515) + * Fixes CVE-CVE-2019-10255 (Closes: #925939) + + -- Gordon Ball <gordon@chronitis.net> Sun, 07 Apr 2019 11:46:04 +0000 + jupyter-notebook (5.7.4-2.1) unstable; urgency=medium * Non-maintainer upload. diff -Nru jupyter-notebook-5.7.4/docs/source/changelog.rst jupyter-notebook-5.7.8/docs/source/changelog.rst --- jupyter-notebook-5.7.4/docs/source/changelog.rst 2018-12-17 10:01:51.000000000 +0000 +++ jupyter-notebook-5.7.8/docs/source/changelog.rst 2019-04-01 10:22:11.000000000 +0000 @@ -21,6 +21,44 @@ Use ``pip install pip --upgrade`` to upgrade pip. Check pip version with ``pip --version``. +.. _release-5.7.8: + +5.7.8 +----- + +- Fix regression in restarting kernels in 5.7.5. + The restart handler would return before restart was completed. +- Further improve compatibility with tornado 6 with improved + checks for when websockets are closed. +- Fix regression in 5.7.6 on Windows where .js files could have the wrong mime-type. +- Fix Open Redirect vulnerability (CVE-2019-10255) + where certain malicious URLs could redirect from the Jupyter login page + to a malicious site after a successful login. + 5.7.7 contained only a partial fix for this issue. + +.. _release-5.7.6: + +5.7.6 +----- + +5.7.6 contains a security fix for a cross-site inclusion (XSSI) vulnerability (CVE-2019–9644), +where files at a known URL could be included in a page from an unauthorized website if the user is logged into a Jupyter server. +The fix involves setting the ``X-Content-Type-Options: nosniff`` +header, and applying CSRF checks previously on all non-GET +API requests to GET requests to API endpoints and the /files/ endpoint. + +The attacking page is able to access some contents of files when using Internet Explorer through script errors, +but this has not been demonstrated with other browsers. + +.. _release-5.7.5: + +5.7.5 +----- + +- Fix compatibility with tornado 6 (:ghpull:`4392`, :ghpull:`4449`). +- Fix opening integer filedescriptor during startup on Python 2 (:ghpull:`4349`) +- Fix compatibility with asynchronous `KernelManager.restart_kernel` methods (:ghpull:`4412`) + .. _release-5.7.4: 5.7.4 diff -Nru jupyter-notebook-5.7.4/notebook/auth/login.py jupyter-notebook-5.7.8/notebook/auth/login.py --- jupyter-notebook-5.7.4/notebook/auth/login.py 2018-12-17 10:01:51.000000000 +0000 +++ jupyter-notebook-5.7.8/notebook/auth/login.py 2019-04-01 10:22:11.000000000 +0000 @@ -7,9 +7,9 @@ import os try: - from urllib.parse import urlparse # Py 3 + from urllib.parse import urlparse, urlunparse # Py 3 except ImportError: - from urlparse import urlparse # Py 2 + from urlparse import urlparse, urlunparse # Py 2 import uuid from tornado.escape import url_escape @@ -39,15 +39,23 @@ """ if default is None: default = self.base_url - if not url.startswith(self.base_url): + # protect chrome users from mishandling unescaped backslashes. + # \ is not valid in urls, but some browsers treat it as / + # instead of %5C, causing `\\` to behave as `//` + url = url.replace("\\", "%5C") + parsed = urlparse(url) + path_only = urlunparse(parsed._replace(netloc='', scheme='')) + if url != path_only or not (parsed.path + '/').startswith(self.base_url): # require that next_url be absolute path within our path allow = False # OR pass our cross-origin check - if '://' in url: + if url != path_only: # if full URL, run our cross-origin check: - parsed = urlparse(url.lower()) origin = '%s://%s' % (parsed.scheme, parsed.netloc) - if self.allow_origin: + origin = origin.lower() + if origin == '%s://%s' % (self.request.protocol, self.request.host): + allow = True + elif self.allow_origin: allow = self.allow_origin == origin elif self.allow_origin_pat: allow = bool(self.allow_origin_pat.match(origin)) diff -Nru jupyter-notebook-5.7.4/notebook/auth/tests/test_login.py jupyter-notebook-5.7.8/notebook/auth/tests/test_login.py --- jupyter-notebook-5.7.4/notebook/auth/tests/test_login.py 1970-01-01 00:00:00.000000000 +0000 +++ jupyter-notebook-5.7.8/notebook/auth/tests/test_login.py 2019-04-01 10:22:11.000000000 +0000 @@ -0,0 +1,54 @@ +"""Tests for login redirects""" + +import requests +from tornado.httputil import url_concat + +from notebook.tests.launchnotebook import NotebookTestBase + + +class LoginTest(NotebookTestBase): + def login(self, next): + first = requests.get(self.base_url() + "login") + first.raise_for_status() + resp = requests.post( + url_concat( + self.base_url() + "login", + {'next': next}, + ), + allow_redirects=False, + data={ + "password": self.token, + "_xsrf": first.cookies.get("_xsrf", ""), + }, + cookies=first.cookies, + ) + resp.raise_for_status() + return resp.headers['Location'] + + def test_next_bad(self): + for bad_next in ( + "//some-host", + "//host" + self.url_prefix + "tree", + "https://google.com";, + "/absolute/not/base_url", + "///jupyter.org", + "/\\some-host", + ): + url = self.login(next=bad_next) + self.assertEqual(url, self.url_prefix) + assert url + + def test_next_ok(self): + for next_path in ( + "tree/", + self.base_url() + "has/host", + "notebooks/notebook.ipynb", + "tree//something", + ): + if "://" in next_path: + expected = next_path + else: + expected = self.url_prefix + next_path + + actual = self.login(next=expected) + self.assertEqual(actual, expected) diff -Nru jupyter-notebook-5.7.4/notebook/base/handlers.py jupyter-notebook-5.7.8/notebook/base/handlers.py --- jupyter-notebook-5.7.4/notebook/base/handlers.py 2018-12-17 10:01:51.000000000 +0000 +++ jupyter-notebook-5.7.8/notebook/base/handlers.py 2019-04-01 10:22:11.000000000 +0000 @@ -82,6 +82,7 @@ def set_default_headers(self): headers = {} + headers["X-Content-Type-Options"] = "nosniff" headers.update(self.settings.get('headers', {})) headers["Content-Security-Policy"] = self.content_security_policy @@ -399,13 +400,69 @@ ) return allow + def check_referer(self): + """Check Referer for cross-site requests. + + Disables requests to certain endpoints with + external or missing Referer. + + If set, allow_origin settings are applied to the Referer + to whitelist specific cross-origin sites. + + Used on GET for api endpoints and /files/ + to block cross-site inclusion (XSSI). + """ + host = self.request.headers.get("Host") + referer = self.request.headers.get("Referer") + + if not host: + self.log.warning("Blocking request with no host") + return False + if not referer: + self.log.warning("Blocking request with no referer") + return False + + referer_url = urlparse(referer) + referer_host = referer_url.netloc + if referer_host == host: + return True + + # apply cross-origin checks to Referer: + origin = "{}://{}".format(referer_url.scheme, referer_url.netloc) + if self.allow_origin: + allow = self.allow_origin == origin + elif self.allow_origin_pat: + allow = bool(self.allow_origin_pat.match(origin)) + else: + # No CORS settings, deny the request + allow = False + + if not allow: + self.log.warning("Blocking Cross Origin request for %s. Referer: %s, Host: %s", + self.request.path, origin, host, + ) + return allow + def check_xsrf_cookie(self): """Bypass xsrf cookie checks when token-authenticated""" if self.token_authenticated or self.settings.get('disable_check_xsrf', False): # Token-authenticated requests do not need additional XSRF-check # Servers without authentication are vulnerable to XSRF return - return super(IPythonHandler, self).check_xsrf_cookie() + try: + return super(IPythonHandler, self).check_xsrf_cookie() + except web.HTTPError as e: + if self.request.method in {'GET', 'HEAD'}: + # Consider Referer a sufficient cross-origin check for GET requests + if not self.check_referer(): + referer = self.request.headers.get('Referer') + if referer: + msg = "Blocking Cross Origin request from {}.".format(referer) + else: + msg = "Blocking request from unknown origin" + raise web.HTTPError(403, msg) + else: + raise def check_host(self): """Check the host header if remote access disallowed. @@ -650,13 +707,20 @@ "; sandbox allow-scripts" @web.authenticated + def head(self, path): + self.check_xsrf_cookie() + return super(AuthenticatedFileHandler, self).head(path) + + @web.authenticated def get(self, path): + self.check_xsrf_cookie() + if os.path.splitext(path)[1] == '.ipynb' or self.get_argument("download", False): name = path.rsplit('/', 1)[-1] self.set_attachment_header(name) return web.StaticFileHandler.get(self, path) - + def get_content_type(self): path = self.absolute_path.strip('/') if '/' in path: diff -Nru jupyter-notebook-5.7.4/notebook/base/zmqhandlers.py jupyter-notebook-5.7.8/notebook/base/zmqhandlers.py --- jupyter-notebook-5.7.4/notebook/base/zmqhandlers.py 2018-12-17 10:01:51.000000000 +0000 +++ jupyter-notebook-5.7.8/notebook/base/zmqhandlers.py 2019-04-01 10:22:11.000000000 +0000 @@ -17,7 +17,8 @@ import tornado from tornado import gen, ioloop, web -from tornado.websocket import WebSocketHandler +from tornado.iostream import StreamClosedError +from tornado.websocket import WebSocketHandler, WebSocketClosedError from jupyter_client.session import Session from jupyter_client.jsonutil import date_default, extract_dates @@ -172,7 +173,7 @@ def send_ping(self): """send a ping to keep the websocket alive""" - if self.stream.closed() and self.ping_callback is not None: + if self.ws_connection is None and self.ping_callback is not None: self.ping_callback.stop() return @@ -185,8 +186,13 @@ self.log.warning("WebSocket ping timeout after %i ms.", since_last_pong) self.close() return + try: + self.ping(b'') + except (StreamClosedError, WebSocketClosedError): + # websocket has been closed, stop pinging + self.ping_callback.stop() + return - self.ping(b'') self.last_ping = now def on_pong(self, data): @@ -237,7 +243,7 @@ def _on_zmq_reply(self, stream, msg_list): # Sometimes this gets triggered when the on_close method is scheduled in the # eventloop but hasn't been called. - if self.stream.closed() or stream.closed(): + if self.ws_connection is None or stream.closed(): self.log.warning("zmq message arrived on closed channel") self.close() return @@ -246,8 +252,14 @@ msg = self._reserialize_reply(msg_list, channel=channel) except Exception: self.log.critical("Malformed message: %r" % msg_list, exc_info=True) - else: + return + + try: self.write_message(msg, binary=isinstance(msg, bytes)) + except (StreamClosedError, WebSocketClosedError): + self.log.warning("zmq message arrived on closed channel") + self.close() + return class AuthenticatedZMQStreamHandler(ZMQStreamHandler, IPythonHandler): @@ -281,7 +293,8 @@ # assign and yield in two step to avoid tornado 3 issues res = self.pre_get() yield gen.maybe_future(res) - super(AuthenticatedZMQStreamHandler, self).get(*args, **kwargs) + res = super(AuthenticatedZMQStreamHandler, self).get(*args, **kwargs) + yield gen.maybe_future(res) def initialize(self): self.log.debug("Initializing websocket connection %s", self.request.path) diff -Nru jupyter-notebook-5.7.4/notebook/files/handlers.py jupyter-notebook-5.7.8/notebook/files/handlers.py --- jupyter-notebook-5.7.4/notebook/files/handlers.py 2018-12-17 10:01:51.000000000 +0000 +++ jupyter-notebook-5.7.8/notebook/files/handlers.py 2019-04-01 10:22:11.000000000 +0000 @@ -35,10 +35,13 @@ @web.authenticated def head(self, path): - self.get(path, include_body=False) + self.check_xsrf_cookie() + return self.get(path, include_body=False) @web.authenticated def get(self, path, include_body=True): + # /files/ requests must originate from the same site + self.check_xsrf_cookie() cm = self.contents_manager if cm.is_hidden(path) and not cm.allow_hidden: diff -Nru jupyter-notebook-5.7.4/notebook/notebookapp.py jupyter-notebook-5.7.8/notebook/notebookapp.py --- jupyter-notebook-5.7.4/notebook/notebookapp.py 2018-12-17 10:01:51.000000000 +0000 +++ jupyter-notebook-5.7.8/notebook/notebookapp.py 2019-04-01 10:22:11.000000000 +0000 @@ -1581,10 +1581,12 @@ def init_mime_overrides(self): # On some Windows machines, an application has registered an incorrect - # mimetype for CSS in the registry. Tornado uses this when serving - # .css files, causing browsers to reject the stylesheet. We know the - # mimetype always needs to be text/css, so we override it here. + # mimetype for CSS and JavaScript in the registry. + # Tornado uses this when serving .css and .js files, causing browsers to + # reject these files. We know the mimetype always needs to be text/css for css + # and application/javascript for JS, so we override it here. mimetypes.add_type('text/css', '.css') + mimetypes.add_type('application/javascript', '.js') def shutdown_no_activity(self): @@ -1739,7 +1741,7 @@ # Write a temporary file to open in the browser fd, open_file = tempfile.mkstemp(suffix='.html') - with open(fd, 'w', encoding='utf-8') as fh: + with io.open(fd, 'w', encoding='utf-8') as fh: self._write_browser_open_file(uri, fh) else: open_file = self.browser_open_file diff -Nru jupyter-notebook-5.7.4/notebook/services/kernels/kernelmanager.py jupyter-notebook-5.7.8/notebook/services/kernels/kernelmanager.py --- jupyter-notebook-5.7.4/notebook/services/kernels/kernelmanager.py 2018-12-17 10:01:51.000000000 +0000 +++ jupyter-notebook-5.7.8/notebook/services/kernels/kernelmanager.py 2019-04-01 10:22:11.000000000 +0000 @@ -280,10 +280,11 @@ self.last_kernel_activity = utcnow() return super(MappingKernelManager, self).shutdown_kernel(kernel_id, now=now) + @gen.coroutine def restart_kernel(self, kernel_id): """Restart a kernel by kernel_id""" self._check_kernel_id(kernel_id) - super(MappingKernelManager, self).restart_kernel(kernel_id) + yield gen.maybe_future(super(MappingKernelManager, self).restart_kernel(kernel_id)) kernel = self.get_kernel(kernel_id) # return a Future that will resolve when the kernel has successfully restarted channel = kernel.connect_shell() @@ -319,7 +320,8 @@ channel.on_recv(on_reply) loop = IOLoop.current() timeout = loop.add_timeout(loop.time() + self.kernel_info_timeout, on_timeout) - return future + # wait for restart to complete + yield future def notify_connect(self, kernel_id): """Notice a new connection to a kernel""" diff -Nru jupyter-notebook-5.7.4/notebook/services/nbconvert/handlers.py jupyter-notebook-5.7.8/notebook/services/nbconvert/handlers.py --- jupyter-notebook-5.7.4/notebook/services/nbconvert/handlers.py 2018-12-17 10:01:51.000000000 +0000 +++ jupyter-notebook-5.7.8/notebook/services/nbconvert/handlers.py 2019-04-01 10:22:11.000000000 +0000 @@ -9,6 +9,7 @@ @web.authenticated def get(self): + self.check_xsrf_cookie() try: from nbconvert.exporters import base except ImportError as e: diff -Nru jupyter-notebook-5.7.4/notebook/static/base/js/namespace.js jupyter-notebook-5.7.8/notebook/static/base/js/namespace.js --- jupyter-notebook-5.7.4/notebook/static/base/js/namespace.js 2018-12-17 10:01:51.000000000 +0000 +++ jupyter-notebook-5.7.8/notebook/static/base/js/namespace.js 2019-04-01 10:22:11.000000000 +0000 @@ -73,7 +73,7 @@ // tree jglobal('SessionList','tree/js/sessionlist'); - Jupyter.version = "5.7.4"; + Jupyter.version = "5.7.8"; Jupyter._target = '_blank'; return Jupyter; }); diff -Nru jupyter-notebook-5.7.4/notebook/utils.py jupyter-notebook-5.7.8/notebook/utils.py --- jupyter-notebook-5.7.4/notebook/utils.py 2018-12-17 10:01:51.000000000 +0000 +++ jupyter-notebook-5.7.8/notebook/utils.py 2019-04-01 10:22:11.000000000 +0000 @@ -13,12 +13,30 @@ from distutils.version import LooseVersion try: + from inspect import isawaitable +except ImportError: + def isawaitable(f): + """If isawaitable is undefined, nothing is awaitable""" + return False + +try: + from concurrent.futures import Future as ConcurrentFuture +except ImportError: + class ConcurrentFuture: + """If concurrent.futures isn't importable, nothing will be a c.f.Future""" + pass + +try: from urllib.parse import quote, unquote, urlparse, urljoin from urllib.request import pathname2url except ImportError: from urllib import quote, unquote, pathname2url from urlparse import urlparse, urljoin +# tornado.concurrent.Future is asyncio.Future +# in tornado >=5 with Python 3 +from tornado.concurrent import Future as TornadoFuture +from tornado import gen from ipython_genutils import py3compat # UF_HIDDEN is a stat flag not defined in the stat module. @@ -306,3 +324,33 @@ check_pid = _check_pid_win32 else: check_pid = _check_pid_posix + + +def maybe_future(obj): + """Like tornado's gen.maybe_future + + but more compatible with asyncio for recent versions + of tornado + """ + if isinstance(obj, TornadoFuture): + return obj + elif isawaitable(obj): + return asyncio.ensure_future(obj) + elif isinstance(obj, ConcurrentFuture): + return asyncio.wrap_future(obj) + else: + # not awaitable, wrap scalar in future + f = TornadoFuture() + f.set_result(obj) + return f + +# monkeypatch tornado gen.maybe_future +# on Python 3 +# TODO: remove monkeypatch after backporting smaller fix to 5.x +try: + import asyncio +except ImportError: + pass +else: + import tornado.gen + tornado.gen.maybe_future = maybe_future diff -Nru jupyter-notebook-5.7.4/notebook/_version.py jupyter-notebook-5.7.8/notebook/_version.py --- jupyter-notebook-5.7.4/notebook/_version.py 2018-12-17 10:01:51.000000000 +0000 +++ jupyter-notebook-5.7.8/notebook/_version.py 2019-04-01 10:22:11.000000000 +0000 @@ -9,5 +9,5 @@ # Next beta/alpha/rc release: The version number for beta is X.Y.ZbN **without dots**. -version_info = (5, 7, 4, '') +version_info = (5, 7, 8) __version__ = '.'.join(map(str, version_info[:3])) + ''.join(version_info[3:]) diff -Nru jupyter-notebook-5.7.4/setup.py jupyter-notebook-5.7.8/setup.py --- jupyter-notebook-5.7.4/setup.py 2018-12-17 10:01:51.000000000 +0000 +++ jupyter-notebook-5.7.8/setup.py 2019-04-01 10:22:11.000000000 +0000 @@ -79,7 +79,7 @@ zip_safe = False, install_requires = [ 'jinja2', - 'tornado>=4', + 'tornado>=4.1,<7', # pyzmq>=17 is not technically necessary, # but hopefully avoids incompatibilities with Tornado 5. April 2018 'pyzmq>=17', diff -Nru jupyter-notebook-5.7.4/.travis.yml jupyter-notebook-5.7.8/.travis.yml --- jupyter-notebook-5.7.4/.travis.yml 2018-12-17 10:01:51.000000000 +0000 +++ jupyter-notebook-5.7.8/.travis.yml 2019-04-01 10:22:11.000000000 +0000 @@ -49,7 +49,8 @@ fi install: - - pip install --pre .[test] + - pip install --pre .[test] $EXTRA_PIP + - pip freeze - wget https://github.com/jgm/pandoc/releases/download/1.19.1/pandoc-1.19.1-1-amd64.deb && sudo dpkg -i pandoc-1.19.1-1-amd64.deb @@ -96,10 +97,19 @@ env: GROUP=python - python: 3.5 env: GROUP=python - - python: "3.7-dev" + - python: 3.7 + dist: xenial env: GROUP=python - python: 3.6 env: GROUP=docs + - python: 3.6 + env: + - GROUP=python + - EXTRA_PIP="tornado<5" + - python: 2.7 + env: + - GROUP=python + - EXTRA_PIP="tornado<5" after_success: - codecov
--- End Message ---
--- Begin Message ---
- To: 926594-done@bugs.debian.org, Gordon Ball <gordon@chronitis.net>
- Subject: Re: unblock: jupyter-notebook/5.7.8-1
- From: Paul Gevers <elbrus@debian.org>
- Date: Fri, 10 May 2019 20:49:12 +0200
- Message-id: <19aa2195-a203-176d-644d-ecc6506d86fd@debian.org>
- In-reply-to: <155464430972.4530.14551433539084466581.reportbug@emissary.chronitis.net>
- References: <155464430972.4530.14551433539084466581.reportbug@emissary.chronitis.net> <155464430972.4530.14551433539084466581.reportbug@emissary.chronitis.net>
Hi Gordon, On Sun, 07 Apr 2019 13:38:29 +0000 Gordon Ball <gordon@chronitis.net> wrote: > unblock jupyter-notebook/5.7.8-1 unblock, thanks. PaulAttachment: signature.asc
Description: OpenPGP digital signature
--- End Message ---