Bug#809211: tracker.debian.org: sqlite backend not usable
[ CCing upstream Django developer to have its opinion ]
Hello,
On Fri, 08 Jan 2016, Christophe Siraut wrote:
> Sorry for the confusion, I have the same libsqlite3 version as you, and
> it is that one I recompiled with SQLITE_MAX_COLUMN 32767.
I can't reproduce the issue with Django 1.8. It looks like a regression of
Django 1.9 that uses an SQL request to generate quoted values for the
parameters fed into requests of the from "WHERE foo IN (...value
list...)".
Also does this also happen if you drop debug_toolbar from INSTALLED_APPS ?
I have the feeling that this code path is only used when you actually want
to print the SQL query that has been executed and this is usally only
needed during development.
The regression has been introduced by this commit:
https://github.com/django/django/commit/4f6a7663bcddffb114f2647f9928cbf1fdd8e4b5
commit 4f6a7663bcddffb114f2647f9928cbf1fdd8e4b5
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Sep 13 09:30:35 2015 +0200
Refs #14091 -- Fixed connection.queries on SQLite.
It's supposed to fix https://code.djangoproject.com/ticket/14091 for
SQLite. Christophe, can you verify that the issue is gone if you revert
that change in your Django ?
Aymeric, the above change is problematic because for a request that
looks like MyModel.objects.filter(foo__in=my_array) it will break as soon
as my_array goes above 2000 entries in length since that's the default
limit for SQLITE_MAX_COLUMN and you will trigger it with your huge
"SELECT QUOTE(?),...." query.
I know the query can also trigger the limit set with
SQLITE_MAX_VARIABLE_NUMBER which defaults to 999 but at least on Debian
we increased that limit significantly (to 250000) so that limit
is not a practical problem. The above limit is more problematic
as we can't increase it to the same value (the max is 32767) and
while the limit on the variable number was unreasonably low, I can
understand the limit on the number of columns much better.
See https://sqlite.org/limits.html
Aymeric, what do you think ? Shall we open a ticket for this regression ?
Can you update the code to process parameters by batch of less than 2000
entries ?
You can look at https://bugs.debian.org/809211 for the former discussion
in this bug report.
Cheers,
--
Raphaël Hertzog ◈ Debian Developer
Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/
Reply to: