Thanks a lot for this!
Matija Nalis <mnalis-debianbug@voyager.hr> writes:
> python-bcrypt and python-itsdangerous on which mediagoblin depends
> are not present in wheezy, so I copied/recompiled them from jessie.
> After that, package compiled and installed without problems. So
> I'd say "installing part" is ready for jessie!
Yeah, I got both of those packages into jessie in anticipation of the
mediagoblin package, but not in time for wheezy.
> Then I proceeded following instructions for manual configuration in
> /usr/share/doc/mediagoblin/README.Debian.gz - here are few issues (on
> wheezy):
>
> - "sudo -u postgres createdb -E UNICODE -O mediagoblin mediagoblin"
> complains with:
> "createdb: database creation failed: ERROR: new encoding (UTF8) is
> incompatible with the encoding of the template database (SQL_ASCII)
> HINT: Use the same encoding as in the template database, or use
> template0 as template."
>
> So I did: "sudo -u postgres createdb -E UNICODE -T template0 -O mediagoblin mediagoblin"
> which didn't complain. Hopefully it did the right thing :)
That's strange. It works for me, and psql -c '\list' shows the encoding
of template1 as UTF8 on both a wheezy machine and a jessie machine. Is
this a fresh install?
> - "su - [mediagoblin]"
> there is no user mediagoblin (maybe it should be created in some step before?
> or package should autocreate it in postinst? it is not mentioned) and I don't
> think square brackets are supposed to be there.
>
> So I did: "useradd --system --home-dir /usr/share/mediagoblin mediagoblin"
> and "su - mediagoblin"
Good point. I've added that to the README.Debian.
> - "Copy /usr/share/mediagoblin/mediagoblin/mediagoblin.ini to /etc/mediagoblin_local.ini"
> well, there wasn't mediagoblin.ini in package at all. Should be fixed.
>
> So I copied it from source tarball from which I built package.
> And, it didn't work as mediagoblin user (from su(1) in step before), so I did as root:
> "cp /home/mnalis/mediagoblin/mediagoblin-0.6.1/mediagoblin.ini /etc/mediagoblin_local.ini"
> "$EDITOR /etc/mediagoblin_local.ini"
> "chown root.mediagoblin /etc/mediagoblin_local.ini"
> "chmod 640 /etc/mediagoblin_local.ini"
I've changed the package to copy mediagoblin.ini into /etc/mediagoblin/, and moved
that section before the part that changes to the mediagoblin user.
> - "$ /usr/bin/gmg dbupdate"
>
> It failed for me:
> Traceback (most recent call last):
> File "/usr/bin/gmg", line 5, in <module>
> exit(main_cli())
> File "/usr/share/mediagoblin/mediagoblin/gmg_commands/__init__.py", line 96, in main_cli
> setup_func = import_component(command_struct['setup'])
> File "/usr/share/mediagoblin/mediagoblin/tools/common.py", line 34, in import_component
> __import__(module_name)
> File "/usr/share/mediagoblin/mediagoblin/gmg_commands/reprocess.py", line 20, in <module>
> from mediagoblin.db.models import MediaEntry
> File "/usr/share/mediagoblin/mediagoblin/db/models.py", line 27, in <module>
> from sqlalchemy.orm import relationship, backref, with_polymorphic
> ImportError: cannot import name with_polymorphic
>
> It seems that "with_polymorphic()" is introduced in sqlalchemy.org 0.8
> release, so we should fix depends version in package to be at least that.
>
> So I upgraded to python-sqlalchemy_0.9.7-1_all.deb from Jessie
I hadn't tested it with sqlalchemy. I've added that dependency.
> that failed with:
> $ /usr/bin/gmg dbupdate
> Traceback (most recent call last):
> File "/usr/bin/gmg", line 5, in <module>
> exit(main_cli())
> File "/usr/share/mediagoblin/mediagoblin/gmg_commands/__init__.py", line 96, in main_cli
> setup_func = import_component(command_struct['setup'])
> File "/usr/share/mediagoblin/mediagoblin/tools/common.py", line 34, in import_component
> __import__(module_name)
> File "/usr/share/mediagoblin/mediagoblin/gmg_commands/reprocess.py", line 20, in <module>
> from mediagoblin.db.models import MediaEntry
> File "/usr/share/mediagoblin/mediagoblin/db/models.py", line 36, in <module>
> from mediagoblin.db.mixin import UserMixin, MediaEntryMixin, \
> File "/usr/share/mediagoblin/mediagoblin/db/mixin.py", line 40, in <module>
> from mediagoblin.tools.text import cleaned_markdown_conversion
> File "/usr/share/mediagoblin/mediagoblin/tools/text.py", line 19, in <module>
> from lxml.html.clean import Cleaner
> ImportError: No module named lxml.html.clean
>
> So I installed "python-lxml" packege from wheezy-backports (which
> wanted to downgrade python-sqlalchemy, but worked around it). So we
> need to depend on it.
It looks like I wasn't seeing that since python-webtest recommends
python-lxml. You're right, it should be a depends.
> that failed with:
> $ /usr/bin/gmg dbupdate
> Traceback (most recent call last):
> File "/usr/bin/gmg", line 5, in <module>
> exit(main_cli())
> File "/usr/share/mediagoblin/mediagoblin/gmg_commands/__init__.py", line 96, in main_cli
> setup_func = import_component(command_struct['setup'])
> File "/usr/share/mediagoblin/mediagoblin/tools/common.py", line 34, in import_component
> __import__(module_name)
> File "/usr/share/mediagoblin/mediagoblin/gmg_commands/reprocess.py", line 21, in <module>
> from mediagoblin.gmg_commands import util as commands_util
> File "/usr/share/mediagoblin/mediagoblin/gmg_commands/util.py", line 18, in <module>
> from mediagoblin import app
> File "/usr/share/mediagoblin/mediagoblin/app.py", line 28, in <module>
> from mediagoblin.tools import common, session, translate, template
> File "/usr/share/mediagoblin/mediagoblin/tools/template.py", line 31, in <module>
> from mediagoblin.tools.timesince import timesince
> File "/usr/share/mediagoblin/mediagoblin/tools/timesince.py", line 32, in <module>
> import pytz
> ImportError: No module named pytz
>
> So I installed "python-tz" (we need to depend on that too)
Same situation, python-babel now depends on it but did not in wheezy.
> that failed with:
> $ /usr/bin/gmg dbupdate
> Traceback (most recent call last):
> File "/usr/bin/gmg", line 5, in <module>
> exit(main_cli())
> File "/usr/share/mediagoblin/mediagoblin/gmg_commands/__init__.py", line 96, in main_cli
> setup_func = import_component(command_struct['setup'])
> File "/usr/share/mediagoblin/mediagoblin/tools/common.py", line 34, in import_component
> __import__(module_name)
> File "/usr/share/mediagoblin/mediagoblin/gmg_commands/reprocess.py", line 22, in <module>
> from mediagoblin.submit.lib import run_process_media
> File "/usr/share/mediagoblin/mediagoblin/submit/lib.py", line 28, in <module>
> from mediagoblin.processing.task import ProcessMedia
> File "/usr/share/mediagoblin/mediagoblin/processing/task.py", line 34, in <module>
> @celery.task(default_retry_delay=2 * 60)
> AttributeError: 'module' object has no attribute 'task'
>
> Ok, it seems python-celery 2.5.3-4 in wheezy is too old. We should
> depend on newer version. Don't know which exactly, but 3.1.12-1
> from jessie seems ok, so I got that (and its few dependencies)
I've changed the depend to need at least 3.1.12-1.
> that failed with:
> $ /usr/bin/gmg dbupdate
> Traceback (most recent call last):
> File "/usr/bin/gmg", line 5, in <module>
> exit(main_cli())
> File "/usr/share/mediagoblin/mediagoblin/gmg_commands/__init__.py", line 112, in main_cli
> args.func(args)
> File "/usr/share/mediagoblin/mediagoblin/gmg_commands/dbupdate.py", line 146, in dbupdate
> run_dbupdate(app_config, global_config)
> File "/usr/share/mediagoblin/mediagoblin/gmg_commands/dbupdate.py", line 119, in run_dbupdate
> run_all_migrations(db, app_config, global_config)
> File "/usr/share/mediagoblin/mediagoblin/gmg_commands/dbupdate.py", line 141, in run_all_migrations
> migration_manager.init_or_migrate()
> File "/usr/share/mediagoblin/mediagoblin/db/migration_tools.py", line 201, in init_or_migrate
> assure_migrations_table_setup(self.session)
> File "/usr/share/mediagoblin/mediagoblin/db/migration_tools.py", line 281, in assure_migrations_table_setup
> if not MigrationData.__table__.exists(db.bind):
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/schema.py", line 638, in exists
> self.name, schema=self.schema)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1732, in run_callable
> with self.contextual_connect() as conn:
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1799, in contextual_connect
> self.pool.connect(),
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 338, in connect
> return _ConnectionFairy._checkout(self)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 641, in _checkout
> fairy = _ConnectionRecord.checkout(pool)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 440, in checkout
> rec = pool._do_get()
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 1055, in _do_get
> return self._create_connection()
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 285, in _create_connection
> return _ConnectionRecord(self)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 411, in __init__
> self.connection = self.__connect()
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 537, in __connect
> connection = self.__pool._creator()
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line 96, in connect
> connection_invalidated=invalidated
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
> reraise(type(exception), exception, tb=exc_tb)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line 90, in connect
> return dialect.connect(*cargs, **cparams)
> File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 377, in connect
> return self.dbapi.connect(*cargs, **cparams)
> sqlalchemy.exc.OperationalError: (OperationalError) unable to open database file None None
>
> some stracing reveals that only looks for mediagoblin_local.ini in current directory.
> so I kludge it with a symlink: "ln -s /etc/mediagoblin_local.ini ~mediagoblin/"
> So probably "/etc/" path got lost somewhere. Should be checked.
>
> anyway, it finally worked, albeit with a warning:
>
> $ /usr/bin/gmg dbupdate
> WARNING:mediagoblin.init.config:When setting up config section, could not import 'mediagoblin.media_types.image'
> -> Initializing main mediagoblin tables... done.
> + Laying foundations for Privilege table
That should be "/usr/bin/gmg -cf /etc/mediagoblin/mediagoblin.ini dbupdate". I'll
think about patching the file that gmg is calling to default to
/etc/mediagoblin/mediagoblin.ini instead of the current directory.
> - "server_name mg.example.com www.mg.example.com" in nginx example config
> is missing ";" at the end of the line
Fixed.
> - starting up mediagoblin fails however with:
> $ PYTHONPATH=/usr/share/mediagoblin/ CELERY_ALWAYS_EAGER=false\
> /usr/bin/paster serve /usr/share/mediagoblin/mediagoblin/paste_local.ini\
> --pid-file=/home/mediagoblin/mediagoblin.pid --server-name=fcgi\
> fcgi_host=127.0.0.1 fcgi_port=26543
> Traceback (most recent call last):
> File "/usr/bin/paster", line 4, in <module>
> command.run()
> File "/usr/lib/python2.7/dist-packages/paste/script/command.py", line 93, in run
> commands = get_commands()
> File "/usr/lib/python2.7/dist-packages/paste/script/command.py", line 135, in get_commands
> plugins = pluginlib.resolve_plugins(plugins)
> File "/usr/lib/python2.7/dist-packages/paste/script/pluginlib.py", line 82, in resolve_plugins
> pkg_resources.require(plugin)
> File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 686, in require
> needed = self.resolve(parse_requirements(requirements))
> File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 584, in resolve
> raise DistributionNotFound(req)
> pkg_resources.DistributionNotFound: Not Found: mediagoblin-0.6.1 (did you run python setup.py develop?)
>
> Now, that might be due to mixing packages from various Debian versions. Oh well :(
> I lack enough working knowledge of python to go past that point ATM.
I've run into this before. It's an issue running it in a directory with
mediagoblin-0.6.1.egg-info in it. I don't totally understand why.
$ pwd
/usr/share/mediagoblin
$ ls
mediagoblin mediagoblin-0.6.1.egg-info
$ PYTHONPATH=/usr/share/mediagoblin/ CELERY_ALWAYS_EAGER=false /usr/bin/paster serve /etc/mediagoblin/paste.ini --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543
Traceback (most recent call last):
File "/usr/bin/paster", line 4, in <module>
command.run()
File "/usr/lib/python2.7/dist-packages/paste/script/command.py", line 93, in run
commands = get_commands()
File "/usr/lib/python2.7/dist-packages/paste/script/command.py", line 135, in get_commands
plugins = pluginlib.resolve_plugins(plugins)
File "/usr/lib/python2.7/dist-packages/paste/script/pluginlib.py", line 82, in resolve_plugins
pkg_resources.require(plugin)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 745, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 639, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: Not Found: mediagoblin-0.6.1 (did you run python setup.py develop?)
$ cd /tmp
$ ls
tmux-0
$ PYTHONPATH=/usr/share/mediagoblin/ CELERY_ALWAYS_EAGER=false /usr/bin/paster serve /etc/mediagoblin/paste.ini --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543
2014-08-09 23:10:55,103 INFO [mediagoblin.app] GNU MediaGoblin 0.6.1 main server starting
2014-08-09 23:10:55,149 INFO [mediagoblin.app] Setting up plugins.
2014-08-09 23:10:55,150 INFO [mediagoblin.init.plugins] Importing plugin module: mediagoblin.plugins.geolocation
2014-08-09 23:10:55,150 INFO [mediagoblin.init.plugins] Importing plugin module: mediagoblin.plugins.basic_auth
2014-08-09 23:10:55,150 INFO [mediagoblin.init.plugins] Importing plugin module: mediagoblin.media_types.image
2014-08-09 23:10:55,191 INFO [mediagoblin.init.celery] Setting celery configuration from object "mediagoblin.init.celery.dummy_settings_module"
Starting server in PID 11258.
^C
$ mkdir mediagoblin-0.6.1.egg-info/
$ PYTHONPATH=/usr/share/mediagoblin/ CELERY_ALWAYS_EAGER=false /usr/bin/paster serve /etc/mediagoblin/paste.ini --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543
Traceback (most recent call last):
File "/usr/bin/paster", line 4, in <module>
command.run()
File "/usr/lib/python2.7/dist-packages/paste/script/command.py", line 93, in run
commands = get_commands()
File "/usr/lib/python2.7/dist-packages/paste/script/command.py", line 135, in get_commands
plugins = pluginlib.resolve_plugins(plugins)
File "/usr/lib/python2.7/dist-packages/paste/script/pluginlib.py", line 82, in resolve_plugins
pkg_resources.require(plugin)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 745, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 639, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: Not Found: mediagoblin-0.6.1 (did you run python setup.py develop?)
I'll look into it further. While I was messing with the paster command I
moved a few things around, and the package now actually has the
paste.ini file (in /etc/mediagoblin/). Putting the file there causes
static files to get placed /etc/mediagoblin/, but I'm working on that as
well.
Again, thanks a lot for going over this package. The updated version
can be found on mentors here:
https://mentors.debian.net/package/mediagoblin
and on collab-maint here:
https://anonscm.debian.org/cgit/collab-maint/mediagoblin.git/
Regards,
Simon
Attachment:
pgprHLzLKIdp4.pgp
Description: PGP signature