--- Begin Message ---
- To: Debian Bug Tracking System <submit@bugs.debian.org>
- Subject: buster-pu: package python-oslo.utils/3.36.4-2 CVE-2019-3866
- From: Thomas Goirand <zigo@debian.org>
- Date: Sat, 21 Dec 2019 22:13:17 +0100
- Message-id: <157696279725.25284.14387113760944954779.reportbug@zbuz.infomaniak.ch>
Package: release.debian.org
Severity: normal
Tags: buster
User: release.debian.org@packages.debian.org
Usertags: pu
Dear Stable release team,
I'd like to update python-oslo.utils in Buster to address CVE-2019-3866.
It wasn't possible to apply directly the patch available here:
https://review.opendev.org/692972
and I found too dangerous to skip the commits right before it, which are
related to this patch. So I just merged upstream branch stable/rocky into
the Debian package. However, looking closer to all patches, either they
are all related to the official patch, or are cosmetic from the Debian
perspective (ie: .gitreview, or upstream CI related).
Please find, attached to this bug, the debdiff for the udpate.
Please allow me to upload:
python-oslo.utils/3.36.4+2019.11.15.git.c49a426b66-1+deb10u1
Cheers,
Thomas Goirand (zigo)
diff -Nru python-oslo.utils-3.36.4/debian/changelog python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/debian/changelog
--- python-oslo.utils-3.36.4/debian/changelog 2018-09-03 23:20:56.000000000 +0200
+++ python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/debian/changelog 2019-12-21 21:37:40.000000000 +0100
@@ -1,3 +1,11 @@
+python-oslo.utils (3.36.4+2019.11.15.git.c49a426b66-1+deb10u1) buster; urgency=medium
+
+ * CVE-2019-3866: Sensitive information leaked in mistral logs. Upgrade to
+ the tip of the upstream stable/rocky branch (as the upstream patch would
+ not otherwise apply).
+
+ -- Thomas Goirand <zigo@debian.org> Sat, 21 Dec 2019 21:37:40 +0100
+
python-oslo.utils (3.36.4-2) unstable; urgency=medium
* Uploading to unstable.
diff -Nru python-oslo.utils-3.36.4/.gitreview python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/.gitreview
--- python-oslo.utils-3.36.4/.gitreview 2018-07-18 05:43:01.000000000 +0200
+++ python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/.gitreview 2019-11-18 10:13:41.000000000 +0100
@@ -1,4 +1,5 @@
[gerrit]
-host=review.openstack.org
+host=review.opendev.org
port=29418
-project=openstack/oslo.utils.git
\ No newline at end of file
+project=openstack/oslo.utils.git
+defaultbranch=stable/rocky
diff -Nru python-oslo.utils-3.36.4/oslo_utils/strutils.py python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/oslo_utils/strutils.py
--- python-oslo.utils-3.36.4/oslo_utils/strutils.py 2018-07-18 05:43:01.000000000 +0200
+++ python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/oslo_utils/strutils.py 2019-11-18 10:13:41.000000000 +0100
@@ -54,11 +54,19 @@
SLUGIFY_HYPHENATE_RE = re.compile(r"[-\s]+")
-# NOTE(flaper87): The following globals are used by `mask_password`
-_SANITIZE_KEYS = ['adminPass', 'admin_pass', 'password', 'admin_password',
+# NOTE(flaper87): The following globals are used by `mask_password` and
+# `mask_dict_password`
+_SANITIZE_KEYS = ['adminpass', 'admin_pass', 'password', 'admin_password',
'auth_token', 'new_pass', 'auth_password', 'secret_uuid',
'secret', 'sys_pswd', 'token', 'configdrive',
- 'CHAPPASSWORD', 'encrypted_key', 'private_key']
+ 'chappassword', 'encrypted_key', 'private_key',
+ 'encryption_key_id', 'fernetkey', 'sslkey', 'passphrase',
+ 'cephclusterfsid', 'octaviaheartbeatkey', 'rabbitcookie',
+ 'cephmanilaclientkey', 'pacemakerremoteauthkey',
+ 'designaterndckey', 'cephadminkey', 'heatauthencryptionkey',
+ 'cephclientkey', 'keystonecredential',
+ 'barbicansimplecryptokek', 'cephrgwkey', 'swifthashsuffix',
+ 'migrationsshkey', 'cephmdskey', 'cephmonkey']
# NOTE(ldbragst): Let's build a list of regex objects using the list of
# _SANITIZE_KEYS we already have. This way, we only have to add the new key
@@ -69,17 +77,18 @@
# NOTE(amrith): Some regular expressions have only one parameter, some
# have two parameters. Use different lists of patterns here.
-_FORMAT_PATTERNS_1 = [r'(%(key)s\s*[=]\s*)[^\s^\'^\"]+']
-_FORMAT_PATTERNS_2 = [r'(%(key)s\s*[=]\s*[\"\'])[^\"\']*([\"\'])',
- r'(%(key)s\s+[\"\'])[^\"\']*([\"\'])',
- r'([-]{2}%(key)s\s+)[^\'^\"^=^\s]+([\s]*)',
- r'(<%(key)s>)[^<]*(</%(key)s>)',
- r'([\"\']%(key)s[\"\']\s*:\s*[\"\'])[^\"\']*([\"\'])',
- r'([\'"][^"\']*%(key)s[\'"]\s*:\s*u?[\'"])[^\"\']*'
+_FORMAT_PATTERNS_1 = [r'(%(key)s[0-9]*\s*[=]\s*)[^\s^\'^\"]+']
+_FORMAT_PATTERNS_2 = [r'(%(key)s[0-9]*\s*[=]\s*[\"\'])[^\"\']*([\"\'])',
+ r'(%(key)s[0-9]*\s+[\"\'])[^\"\']*([\"\'])',
+ r'([-]{2}%(key)s[0-9]*\s+)[^\'^\"^=^\s]+([\s]*)',
+ r'(<%(key)s[0-9]*>)[^<]*(</%(key)s[0-9]*>)',
+ r'([\"\']%(key)s[0-9]*[\"\']\s*:\s*[\"\'])[^\"\']*'
+ '([\"\'])',
+ r'([\'"][^"\']*%(key)s[0-9]*[\'"]\s*:\s*u?[\'"])[^\"\']*'
'([\'"])',
- r'([\'"][^\'"]*%(key)s[\'"]\s*,\s*\'--?[A-z]+\'\s*,\s*u?'
- '[\'"])[^\"\']*([\'"])',
- r'(%(key)s\s*--?[A-z]+\s*)\S+(\s*)']
+ r'([\'"][^\'"]*%(key)s[0-9]*[\'"]\s*,\s*\'--?[A-z]+'
+ '\'\s*,\s*u?[\'"])[^\"\']*([\'"])',
+ r'(%(key)s[0-9]*\s*--?[A-z]+\s*)\S+(\s*)']
# NOTE(dhellmann): Keep a separate list of patterns by key so we only
# need to apply the substitutions for keys we find using a quick "in"
@@ -89,11 +98,11 @@
_SANITIZE_PATTERNS_2[key] = []
for pattern in _FORMAT_PATTERNS_2:
- reg_ex = re.compile(pattern % {'key': key}, re.DOTALL)
+ reg_ex = re.compile(pattern % {'key': key}, re.DOTALL | re.IGNORECASE)
_SANITIZE_PATTERNS_2[key].append(reg_ex)
for pattern in _FORMAT_PATTERNS_1:
- reg_ex = re.compile(pattern % {'key': key}, re.DOTALL)
+ reg_ex = re.compile(pattern % {'key': key}, re.DOTALL | re.IGNORECASE)
_SANITIZE_PATTERNS_1[key].append(reg_ex)
@@ -328,7 +337,7 @@
# specified in _SANITIZE_KEYS, if not then just return the message since
# we don't have to mask any passwords.
for key in _SANITIZE_KEYS:
- if key in message:
+ if key.lower() in message.lower():
for pattern in _SANITIZE_PATTERNS_2[key]:
message = re.sub(pattern, substitute2, message)
for pattern in _SANITIZE_PATTERNS_1[key]:
@@ -406,7 +415,7 @@
k_matched = False
if isinstance(k, six.string_types):
for sani_key in _SANITIZE_KEYS:
- if sani_key in k:
+ if sani_key.lower() in k.lower():
out[k] = secret
k_matched = True
break
diff -Nru python-oslo.utils-3.36.4/oslo_utils/tests/test_strutils.py python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/oslo_utils/tests/test_strutils.py
--- python-oslo.utils-3.36.4/oslo_utils/tests/test_strutils.py 2018-07-18 05:43:01.000000000 +0200
+++ python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/oslo_utils/tests/test_strutils.py 2019-11-18 10:13:41.000000000 +0100
@@ -352,6 +352,43 @@
payload = """{ 'token' : 'token' }"""
expected = """{ 'token' : '***' }"""
self.assertEqual(expected, strutils.mask_password(payload))
+ # Test 'fernetkey'
+ payload = """{ 'fernetkey' : 'token' }"""
+ expected = """{ 'fernetkey' : '***' }"""
+ self.assertEqual(expected, strutils.mask_password(payload))
+ # Test 'FernetKey'
+ payload = """{ 'FernetKey' : 'token' }"""
+ expected = """{ 'FernetKey' : '***' }"""
+ self.assertEqual(expected, strutils.mask_password(payload))
+ # Test 'sslkey'
+ payload = """{ 'sslkey' : 'token' }"""
+ expected = """{ 'sslkey' : '***' }"""
+ self.assertEqual(expected, strutils.mask_password(payload))
+ # Test 'SslKey'
+ payload = """{ 'SslKey' : 'token' }"""
+ expected = """{ 'SslKey' : '***' }"""
+ self.assertEqual(expected, strutils.mask_password(payload))
+ # Test 'passphrase'
+ payload = """{ 'passphrase' : 'token' }"""
+ expected = """{ 'passphrase' : '***' }"""
+ self.assertEqual(expected, strutils.mask_password(payload))
+ # Test 'PassPhrase'
+ payload = """{ 'PassPhrase' : 'token' }"""
+ expected = """{ 'PassPhrase' : '***' }"""
+ self.assertEqual(expected, strutils.mask_password(payload))
+ # Some real-life cases
+ # Test 'KeystoneFernetKey1'
+ payload = """{ 'KeystoneFernetKey1' : 'token' }"""
+ expected = """{ 'KeystoneFernetKey1' : '***' }"""
+ self.assertEqual(expected, strutils.mask_password(payload))
+ # Test 'OctaviaCaKeyPassword'
+ payload = """{ 'OctaviaCaKeyPassword' : 'token' }"""
+ expected = """{ 'OctaviaCaKeyPassword' : '***' }"""
+ self.assertEqual(expected, strutils.mask_password(payload))
+ # Test 'OctaviaCaKeyPassphrase'
+ payload = """{ 'OctaviaCaKeyPassphrase' : 'token' }"""
+ expected = """{ 'OctaviaCaKeyPassphrase' : '***' }"""
+ self.assertEqual(expected, strutils.mask_password(payload))
def test_xml(self):
# Test 'adminPass' w/o spaces
@@ -394,6 +431,10 @@
</password>"""
expected = """<password>***</password>"""
self.assertEqual(expected, strutils.mask_password(payload))
+ # Test 'Password1' - case-insensitive + number
+ payload = """<Password1>TL0EfN33</Password1>"""
+ expected = """<Password1>***</Password1>"""
+ self.assertEqual(expected, strutils.mask_password(payload))
def test_xml_attribute(self):
# Test 'adminPass' w/o spaces
@@ -458,6 +499,9 @@
payload = """body: {"rescue": {"password": "1234567"}}"""
expected = """body: {"rescue": {"password": "***"}}"""
self.assertEqual(expected, strutils.mask_password(payload))
+ payload = """body: {"rescue": {"encryption_key_id": "1234567"}}"""
+ expected = """body: {"rescue": {"encryption_key_id": "***"}}"""
+ self.assertEqual(expected, strutils.mask_password(payload))
def test_xml_message(self):
payload = """<?xml version="1.0" encoding="UTF-8"?>
@@ -647,6 +691,16 @@
self.assertEqual(expected,
strutils.mask_dict_password(payload))
+ payload = {'passwords': {'KeystoneFernetKey1': 'c5FijjS'}}
+ expected = {'passwords': {'KeystoneFernetKey1': '***'}}
+ self.assertEqual(expected,
+ strutils.mask_dict_password(payload))
+
+ payload = {'passwords': {'keystonecredential0': 'c5FijjS'}}
+ expected = {'passwords': {'keystonecredential0': '***'}}
+ self.assertEqual(expected,
+ strutils.mask_dict_password(payload))
+
def test_do_no_harm(self):
payload = {}
expected = {}
diff -Nru python-oslo.utils-3.36.4/releasenotes/notes/mask-dict-passwords-99357ffb7972fb0b.yaml python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/releasenotes/notes/mask-dict-passwords-99357ffb7972fb0b.yaml
--- python-oslo.utils-3.36.4/releasenotes/notes/mask-dict-passwords-99357ffb7972fb0b.yaml 1970-01-01 01:00:00.000000000 +0100
+++ python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/releasenotes/notes/mask-dict-passwords-99357ffb7972fb0b.yaml 2019-11-18 10:13:41.000000000 +0100
@@ -0,0 +1,9 @@
+---
+security:
+ - |
+ This patch ensures that we mask sensitive data when masking dicts, even if
+ the case doesn't match. This means the behaviour of mask_password and
+ mask_dict_password is now the same.
+ - |
+ Additional password names were included from real world logs that contained
+ sensitive information.
\ No newline at end of file
diff -Nru python-oslo.utils-3.36.4/releasenotes/notes/mask-password-patterns-f41524069b8ae488.yaml python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/releasenotes/notes/mask-password-patterns-f41524069b8ae488.yaml
--- python-oslo.utils-3.36.4/releasenotes/notes/mask-password-patterns-f41524069b8ae488.yaml 1970-01-01 01:00:00.000000000 +0100
+++ python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/releasenotes/notes/mask-password-patterns-f41524069b8ae488.yaml 2019-11-18 10:13:41.000000000 +0100
@@ -0,0 +1,8 @@
+---
+security:
+ - This patch ensures we actually mask sensitive data, even if case doesn't
+ match the static entry we have in the patterns.
+ - It also ensures that some fancy names with a common base, but added
+ number are actually taken care of.
+fixes:
+ - https://bugs.launchpad.net/tripleo/+bug/1850843
diff -Nru python-oslo.utils-3.36.4/tox.ini python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/tox.ini
--- python-oslo.utils-3.36.4/tox.ini 2018-07-18 05:43:01.000000000 +0200
+++ python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/tox.ini 2019-11-18 10:13:41.000000000 +0100
@@ -5,7 +5,7 @@
[testenv]
install_command = pip install {opts} {packages}
deps =
- -c{env:UPPER_CONSTRAINTS_FILE:https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt}
+ -c{env:UPPER_CONSTRAINTS_FILE:https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt?h=stable/rocky}
-r{toxinidir}/test-requirements.txt
-r{toxinidir}/requirements.txt
commands = stestr run --slowest {posargs}
diff -Nru python-oslo.utils-3.36.4/.zuul.yaml python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/.zuul.yaml
--- python-oslo.utils-3.36.4/.zuul.yaml 2018-07-18 05:43:01.000000000 +0200
+++ python-oslo.utils-3.36.4+2019.11.15.git.c49a426b66/.zuul.yaml 2019-11-18 10:13:41.000000000 +0100
@@ -5,3 +5,11 @@
gate:
jobs:
- openstack-tox-lower-constraints
+ templates:
+ - openstack-python-jobs
+ - openstack-python35-jobs
+ - publish-openstack-docs-pti
+ - check-requirements
+ - lib-forward-testing
+ - release-notes-jobs-python3
+ - periodic-stable-jobs
--- End Message ---