help! broken setup.py install on rampage
i need help with a setup.py of a package i try to crate.
it seems severely broken and i dont know the proper way to fix
it.
I got cvs write access and the license to kill (the setup.py).
my problem is that i hardly know the first thing about setup.pys.
>From the head of the project:
On hard setup.py requirements; it should work with python2.2 (and
newer) distutils, should be able to write cerebrum_path.py to a
directory in the default sys.path, should be able to install
Cerebrum's files as documented in the top-comment of the current
setup.py, and should not be as much of a hack as the current
implementation. Other command-line options (besides --root, which
the debian build process uses), like --prefix, needs to work "as
expected" as well
if you think this can be acived with current python
distutils in a clean way i would welcome tips and help.
you can look at the present setup.py and the mentioned
file-distribution here:
http://cvs.sourceforge.net/viewcvs.py/cerebrum/cerebrum/setup.py
of cause you can check out the source anomymously and try my
totally broken but building package, which is included in that
repository.
I include two bits from my mail exchange with upstream (with
permission):
============== 1.st mail ==================
> setup.py seems to create a file Cerebrum/__init__.pyc when called
> like this:
>
> python setup.py clean
In fact, Cerebrum.__init__.pyc is created whenever setup.py is called
(regardless of other command line arguments), due to the following
line near the top of setup.py:
import Cerebrum
Setup.py needs to import this package in order to discover the current
Cerebrum version number.
> as far as i understand it it should *clean* the sourcetree, not
> leaving compiled files behind? (c:
That would be optimal, yes. However, I'm not sure how this should be
fixed, nor am I sure exactly why it is important to fix it...
> then there is a whole bunch of files which seem to be installed
> only if the user cerebrum exists, as the cerebrum user. these files are
>
> /usr/./share/doc/cerebrum/design/bofhd_auth.sql
> /usr/./share/doc/cerebrum/design/bofhd_tables.sql
> /usr/./share/doc/cerebrum/design/core_tables.sql
> /usr/./share/doc/cerebrum/design/mod_ad.sql
> /usr/./share/doc/cerebrum/design/mod_changelog.sql
> /usr/./share/doc/cerebrum/design/mod_email.sql
> /usr/./share/doc/cerebrum/design/mod_job_runner.sql
> /usr/./share/doc/cerebrum/design/mod_mount_host.sql
> /usr/./share/doc/cerebrum/design/mod_notur_user.sql
> /usr/./share/doc/cerebrum/design/mod_password_history.sql
> /usr/./share/doc/cerebrum/design/mod_person_external_id_history.sql
> /usr/./share/doc/cerebrum/design/mod_person_name_history.sql
> /usr/./share/doc/cerebrum/design/mod_posix_user.sql
> /usr/./share/doc/cerebrum/design/mod_printer_accounting.sql
> /usr/./share/doc/cerebrum/design/mod_printerquotas.sql
> /usr/./share/doc/cerebrum/design/mod_stedkode.sql
> /usr/./share/doc/cerebrum/design/mod_country_alias.sql
> /usr/./share/doc/cerebrum/design/mod_feidegvs.sql
> /usr/./share/doc/cerebrum/cerebrum-core.dia
> /usr/./share/doc/cerebrum/cerebrum-core.html
> /usr/./share/doc/cerebrum/adminprotocol.html
> /usr/./share/doc/cerebrum/README
> /usr/./share/doc/cerebrum/COPYING
> /usr/./sbin/bofhd.py
> /usr/./sbin/job_runner.py
> /usr/./sbin/makedb.py
> /usr/./share/cerebrum/contrib/adquicksync.py
> /usr/./share/cerebrum/contrib/adsync.py
> /usr/./share/cerebrum/contrib/adutils.py
> /usr/./share/cerebrum/contrib/generate_cyrus_folders.py
> /usr/./share/cerebrum/contrib/generate_ldif.py
> /usr/./share/cerebrum/contrib/generate_mail_ldif.py
> /usr/./share/cerebrum/contrib/generate_nismaps.py
> /usr/./share/cerebrum/contrib/import_IMS_data.py
> /usr/./share/cerebrum/contrib/notesquick.py
> /usr/./share/cerebrum/contrib/notesutils.py
> /usr/./share/cerebrum/contrib/nwsync.py
> /usr/./share/cerebrum/contrib/nwutils.py
> /usr/./share/cerebrum/contrib/generate_mail_dns_ldif.py
> /usr/./share/cerebrum/contrib/ldapqsync.py
> /usr/./share/cerebrum/contrib/no/import_SATS.py
> /usr/./share/cerebrum/contrib/no/import_from_MSTAS.py
> /usr/./share/cerebrum/contrib/no/pop_ad_tbl.py
> /usr/./share/cerebrum/contrib/no/update_FS_mailadr.py
> /usr/./share/cerebrum/contrib/no/uio/changelog_examples.py
> /usr/./share/cerebrum/contrib/no/uio/dump_to_UA.py
> /usr/./share/cerebrum/contrib/no/uio/generate_fronter_export.py
> /usr/./share/cerebrum/contrib/no/uio/import_FS.py
> /usr/./share/cerebrum/contrib/no/uio/import_LT.py
> /usr/./share/cerebrum/contrib/no/uio/import_OU.py
> /usr/./share/cerebrum/contrib/no/uio/import_from_FS.py
> /usr/./share/cerebrum/contrib/no/uio/import_from_LT.py
> /usr/./share/cerebrum/contrib/no/uio/import_userdb_XML.py
> /usr/./share/cerebrum/contrib/no/uio/merge_xml_files.py
> /usr/./share/cerebrum/contrib/no/uio/populate_fronter_groups.py
> /usr/./share/cerebrum/contrib/no/uio/pq.py
> /usr/./share/cerebrum/contrib/no/uio/process_bofhd_requests.py
> /usr/./share/cerebrum/contrib/no/uio/process_changes.py
> /usr/./share/cerebrum/contrib/no/uio/process_students.py
> /usr/./share/cerebrum/contrib/no/uio/run_privileged_command.py
> /usr/./share/cerebrum/contrib/no/uio/uio_migrate.conf.py
> /usr/./share/cerebrum/contrib/no/uio/update_LT_mailadr.py
> /usr/./share/cerebrum/contrib/no/uio/import_uio_globals_XML.py
> /usr/./share/cerebrum/contrib/no/uio/make_home2spool.py
> /usr/./share/cerebrum/contrib/no/uio/dbfg_update.py
> /usr/./share/cerebrum/contrib/no/uio/generate_frida_export.py
> /usr/./share/cerebrum/contrib/no/uio/generate_portal_export.py
> /usr/./share/cerebrum/contrib/no/uio/pq_update.py
> /usr/./share/cerebrum/contrib/no/uio/subscribe_imap.py
> /usr/./share/cerebrum/contrib/no/uio/update_employee_groups.py
> /usr/./share/cerebrum/contrib/no/uio/get_kurs_evu_kurs.py
> /usr/./share/cerebrum/contrib/no/uio/get_uname_emnekode.py
> /usr/./share/cerebrum/contrib/no/uio/get_uname_evu_kurs.py
> /usr/./share/cerebrum/contrib/no/uio/ifi_auto.py
> /usr/./share/cerebrum/contrib/no/uio/join_persons.py
> /usr/./share/cerebrum/contrib/no/uio/mailman.py
> /usr/./bin/bofh.py
> /usr/./bin/fix_jbofh_jar.py
> /usr/./share/cerebrum/client/passweb.py
> /usr/./share/cerebrum/client/passweb_form.html
> /usr/./share/cerebrum/client/passweb_receipt.html
> /usr/./etc/cerebrum/cereconf.py
> /usr/./etc/cerebrum/config.dat
> /usr/./etc/cerebrum/scheduled_jobs.py
> /usr/./etc/cerebrum/logging.ini
>
> what is the rational for these files to be owned by user
> cerebrum?
If you haven't already noticed, large parts of Cerebrum's setup.py
script is rather much of a hack. The files you list above are
installed as part of the "data" installation phase, even though they
clearly are "source".
The reason for this hack is a combination of 1) us Cerebrum developers
not being much of Python distutils hackers and 2) convincing Python
distutils to install scripts in more than one directory being damn
cumbersome, at best.
As a result, we ended up going down the path of least resistance,
towards a solution that is "good enough" for the uses we envisioned
ourselves.
I suspect your work on making debian packages of Cerebrum is pushing
the limits of that "good enough" solution -- and while I happen to
think that it would be swell to get a *proper* setup.py in place, I
also think the problems needs new eyes looking at it, rather than us
"old-timers" trying to extend the current hack further.
> i would have thought it would be better to let root own most of
> them.
Could be. The design decision that Cerebrum should (mostly) run under
a non-root UID can be seen as being independent of what UID owns the
installed files.
However, as things are currently standing here at UiO, we regularly
update (parts of) the installation from CVS. I suspect we would see
requiring root privilege for that operation as mostly cumbersome.
The fact that the user name 'cerebrum' is hardcoded and
non-overridable is likely yet another consequence of the hackish
nature of the current setup.py, though, and we would be better off it
was fixed.
==============================================================
==========================2nd mail ===========================
[Andreas Schuldei]
> in line 205 of setup.py the prefix stuff starts:
>
> prefix="." # Should preferably be initialized from the command-line argument
> sharedir="%s/share" % prefix
> sbindir="%s/sbin" % prefix
> bindir="%s/bin" % prefix
> sysconfdir = "%s/etc/cerebrum" % prefix # Should be /etc/cerebrum/
> logdir = "%s/var/log/cerebrum" % prefix # Should be /var/log/cerebrum/
>
> here the installation paths are defined where stuff should go.
Uhm, well, actually there is quite a bit of magic in the various
distutils modules that partake in the decision; the above definitions
are just part of the picture.
> somewhere (not sure where) usr is prepended.
This happens somewhere inside distutils; actually, it is "sys.path"
that is prepended.
> This results in install-paths like this for me:
>
> [...]
> creating /home/andreas/src/deb/cerebrum-0.0.20040131/debian/tmp/usr/etc/cerebrum
> copying design/cereconf.py -> /home/andreas/src/deb/cerebrum-0.0.20040131/debian/tmp/usr/./etc/cerebrum
> [...]
Darn. Another unwanted consequence of Cerebrum's installation using
"install_data" for the rather more than we should, is that if one
tries to do
python setup.py install --install-data=/ --root=/foo
one will end up with cereconf.py in /foo/etc/cerebrum/cereconf.py --
but will also get lots of other stuff outside /foo/usr,
e.g. /foo/sbin/bofhd.py.
> is there a reason for this?
This was not our intention, no.
> how can i get rid of it? are there intentions to change this?
Yet again, I'm afraid that the answer is "fix the current setup.py" --
and I suspect that we here at UiO won't have time to work on that
until we've reduced the amount of more pressing issues by quite a bit.
> i suggest to add usr to the first three dirs statically and stop
> adding /usr automatically to all.
You're of course welcome to have a look inside the distutils magic to
see if that's easily doable.
======================================================================
Reply to: