dak.py init-dirs fails, both as user dak and as user root

To: "debian-dak@lists.debian.org" <debian-dak@lists.debian.org>
Subject: dak.py init-dirs fails, both as user dak and as user root
From: "Brooke, Simon" <Simon.Brooke@henryschein.co.uk>
Date: Wed, 4 Jan 2012 11:46:48 -0500
Message-id: <[🔎] 20120104164827.C098D13A7837@liszt.debian.org>
Hi all

I've been trying to set up a dak server following the READMEs; I think I've done everything right.

However, when I try init-dirs as user dak I get a fail:

dak@debserver:~$ dak.py init-dirs
Creating /srv/dak/ftp/ ...
Creating /srv/dak/ftp/pool/ ...
Creating /srv/dak/dak/templates/ ...
Creating /src/dak/scripts/override/ ...
Traceback (most recent call last):
  File "/srv/dak/bin/dak.py", line 237, in <module>
    main()
  File "/srv/dak/bin/dak.py", line 217, in main
    module.main()
  File "/usr/local/dak/dak/init_dirs.py", line 191, in main
    create_directories()
  File "/usr/local/dak/dak/init_dirs.py", line 111, in create_directories
    process_tree(Cnf, "Dir")
  File "/usr/local/dak/dak/init_dirs.py", line 70, in process_tree
    do_dir(target, config_name)
  File "/usr/local/dak/dak/init_dirs.py", line 54, in do_dir
    os.makedirs(target)
  File "/usr/lib/python2.6/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.6/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.6/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.6/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/src'

When I try this as root I get

root@debserver:/srv/dak# bin/dak.py init-dirs
E: Cannot connect to database ((OperationalError) FATAL:  role "root" does not exist
 None None)

Which is reasonable since one really does not want root interfering with the database.

If I create the appropriate directory as root:

root@debserver:/srv/dak# mkdir -p /src/dak
root@debserver:/srv/dak# chown dak:ftpmaster /src/dak

and then try again as user dak, it gets a bit further but still fails:

dak@debserver:~$ dak.py init-dirs
Creating /src/dak/scripts/override/ ...
Creating /srv/dak/database/dists/ ...
Creating /srv/dak/lock/ ...
Creating /srv/dak/morgue/ ...
Creating /srv/dak/queue/holding/ ...
Creating /srv/dak/queue/done/ ...
Creating  ...
Traceback (most recent call last):
  File "/srv/dak/bin/dak.py", line 237, in <module>
    main()
  File "/srv/dak/bin/dak.py", line 217, in main
    module.main()
  File "/usr/local/dak/dak/init_dirs.py", line 191, in main
    create_directories()
  File "/usr/local/dak/dak/init_dirs.py", line 111, in create_directories
    process_tree(Cnf, "Dir")
  File "/usr/local/dak/dak/init_dirs.py", line 70, in process_tree
    do_dir(target, config_name)
  File "/usr/local/dak/dak/init_dirs.py", line 54, in do_dir
    os.makedirs(target)
  File "/usr/lib/python2.6/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 2] No such file or directory: ''

I'm assuming there's some missing parent path part in this; I don't want a '/src/' in the system root, and guess it ought to have been /srv/dak/src (or perhaps /usr/local/dak/src). Looking further I find there's a path declared in /etc/dak/dak.conf:

    //// Override (optional): This directory optionally stores override files (used
    //// by 'dak make-overrides').
    Override "/src/dak/scripts/override/";

This sort of implies that the empty path at which it is now failing is the next to be declared in /etc/dak/dak.conf *after* Creating /srv/dak/queue/done/; examining the next thing I've got is a commented out line:

    //// BTSVersionTrack (optional): this directory holds the DebBugs
    //// Version Tracking support files.
    // BTSVersionTrack "/srv/dak/btsversiontrack";

However if I experimentally uncomment that it creates one more directory and fails as before:

dak@debserver:~$ dak.py init-dirs
Creating /usr/local/dak/src/dak/scripts/override/ ...
Creating /srv/dak/btsversiontrack ...
Creating  ...
Traceback (most recent call last):
  File "/srv/dak/bin/dak.py", line 237, in <module>
    main()
  File "/srv/dak/bin/dak.py", line 217, in main
    module.main()
  File "/usr/local/dak/dak/init_dirs.py", line 191, in main
    create_directories()
  File "/usr/local/dak/dak/init_dirs.py", line 111, in create_directories
    process_tree(Cnf, "Dir")
  File "/usr/local/dak/dak/init_dirs.py", line 70, in process_tree
    do_dir(target, config_name)
  File "/usr/local/dak/dak/init_dirs.py", line 54, in do_dir
    os.makedirs(target)
  File "/usr/lib/python2.6/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 2] No such file or directory: ''

The next obvious thing in /etc/dak/dak.conf is the Queue section, which I have as:

    Queue
    {
        Byhand "/srv/dak/queue/byhand/";
        New "/srv/dak/queue/new/";
        NewStage "/srv/dak/queue/newstage/";
        Reject "/srv/dak/queue/reject/";
        Unchecked "/srv/dak/queue/unchecked/";
    };

Without knowing anything about how the system is supposed to work, that seems fairly sane.

Nothing else in /etc/dak/dak.conf looks as if it is a missing file path. In order to help people understand this, here is the whole of my current /etc/dak/dak.conf

--------------- /etc/dak/dak.conf begins --------------------
///////////////////////////////////////////////////////////
// Experimental dak configuration file edited for use at
// Henry Schein Animal Health. Simon Brooke 20111230
///////////////////////////////////////////////////////////

///////////////////////////////////////////////////////////
// Common settings
///////////////////////////////////////////////////////////
Common
{
    //// ThreadCount (optional):  Maximum number of threads to use.  Defaults to 1
    ThreadCount 1;
};

///////////////////////////////////////////////////////////
// Essential.  List of database details.
///////////////////////////////////////////////////////////
DB
{
    //// Name (required): The name of the PostgreSQL database which has been
    //// created for dak.
    Name "dak";

    //// Host (optional): The name of the host on which the database is located.
    //// Not necessary if the the database is local.
    //Host "";

    //// Port (optional): The port of the database if it is remote.  If the port
    //// is the default value (5432), this option can be set to -1.
    //Port -1;

    //// PoolSize (optional): should be at least ThreadCount + 1
    PoolSize 5;

    //// MaxOverflow (optional): shouldn't exceed postgresql.conf's
    //// max_connections - PoolSize
    MaxOverflow 13;

    //// Unicode (optional but seriously suggested).  Should be false for
    //// encoding == SQL_ASCII which is highly recommended.  Do not set this to
    //// anything else unless you really know what you're doing.
    Unicode "false";
};

///////////////////////////////////////////////////////////
// Dinstall general settings
///////////////////////////////////////////////////////////
Dinstall
{
    //// SigningKeyring (optional): this is the private keyring used by
    //// 'dak generate-releases'.
    SigningKeyring "/srv/dak/s3kr1t/dot-gnupg/secring.gpg";

    //// SigningPubKeyring (optional): this is the public keyring used by
    //// 'dak generate-releases'.
    SigningPubKeyring "/srv/dak/s3kr1t/dot-gnupg/pubring.gpg";

    //// Options::No-Mail (optional): Highly recommended.  This prevents dak
    //// from spamming people you didn't mean it to.  If you're not using this,
    //// you almost certainly want to use the MailWhiteList option
    Options::No-Mail "false";

    //// MailWhiteList (optional).  If you're not using No-Mail, this provides
    //// a list of email addresses or regular expressions, one per line which
    //// dak is allowed to email.  Regular expressions need to be prefixed by "RE:".
    //// Examples: "jane.doe@domain.com" or "RE:jane[^@]@domain.com", where the first will
    //// only allow to mail jane.doe@domain.com while the second will mail all of jane*@domain.com
    MailWhiteList "/etc/dak/mail.whitelist";

    //// SendmailCommand (required unless No-Mail is set): command to call the MTA.
    SendmailCommand "/usr/sbin/sendmail -oi -t";

    //// MyEmailAddress (required): this is used as the From: line for sending mails
    //// as a script/daemon.
    MyEmailAddress "Software package archive <devserver@eu.hsi.local>";

    //// MyAdminAddress (required): used as a contact address in mails.
    MyAdminAddress "Simon.Brooke@henryschein.co.uk";

    //// MyDistribution (required): this variable is used in emails sent out by
    //// dak and others.  It should indicate the name of the distribution.
    MyDistribution "HSI-Debian";

    //// BugServer (optional): is used by 'dak process-upload' and 'dak rm'
    //// when closing bugs.
    // BugServer "bugs.example.org";

    //// CloseBugs (optional): a boolean (default: no); if true the automated bug
    //// closing feature of dinstall is activated.  Must have BugServer set
    //// for this to work.
    CloseBugs "true";

    //// PackagesServer (optional): used by 'dak rm' and 'dak override' when
    //// carbon-copying a bug close mail to a package maintainer.
    // PackagesServer "packages.example.org";

    //// TrackingServer (optional): used by 'dak process-upload' and 'dak
    //// rm' to send messages for the maintainer also to an alias for people
    //// tracking a specific source package.
    // TrackingServer "packages.qa.example.org";

    //// All sent mail is blind carbon copied to the email address in Bcc if it's
    //// not blank.
    // Bcc "archive@example.org";

    //// BXANotify (optional): a boolean (default: false); if true (Debian-specific)
    //// BXA notification is sent.  The template for the BXA notification is located
    //// in Dir::Templates/process-new.bxa_notification and should be changed if this
    //// option is set.
    // BXANotify "true";

    //// FutureTimeTravelGrace (required): specifies how many seconds into the
    //// future timestamps are allowed to be inside a deb before being rejected.
    //// 28800 = 8 hours
    FutureTimeTravelGrace 28800;

    //// PastCutoffYear (required): specifies the cut-off year which is used when
    //// deciding whether or not to reject packages based on the file timestamp.
    PastCutoffYear "1984";

    //// SkipTime (required): an integer value which is the number of seconds
    //// that a file must be older than (via it's last modified timestamp)
    //// before dak process-upload will REJECT rather than SKIP the package.
    SkipTime 300;

    //// OverrideDisparityCheck (optional): a boolean (default: false); if true,
    //// dak process-upload compares an uploads section/priority with the overrides and whines
    //// at the maintainer if they differ.
    OverrideDisparityCheck "false";

    //// OverrideMaintainer (optional): be used to globally override the
    //// __MAINTAINER_TO__ and __MAINTAINER_FROM__ variables in template mails.
    //// Use with caution.
    // OverrideMaintainer "";

    //// DefaultSuite (optional): which suite to default to where one isn't
    //// specified.  Defaults to "unstable" if not set.
    DefaultSuite "stable";

    //// LintianTags (optional): A set of lintian tags on which to reject
    //// packages at upload time.  Format is a YAML file; see the ftp-master
    //// version for an example.
    // LintianTags "/srv/dak/dak/config/debian/lintian.tags";

    //// ReleaseTransitions (optional): YAML File for blocking uploads to unstable
    // ReleaseTransitions "/srv/dak/web/transitions.yaml";

    //// KeyAutoFetch (optional): boolean (default: false), which if set (and
    //// not overriden by explicit argument to check_signature()) will enable
    //// auto key retrieval.  Requires KeyServer variable be
    //// set.  NB: you should only enable this variable on production systems
    //// if you have strict control of your upload queue.
    // KeyAutoFetch "false";

    //// KeyServer (optional): keyserver used for key auto-retrieval
    //// (c.f. KeyAutoFetch).
    // KeyServer "wwwkeys.eu.pgp.net";
};


///////////////////////////////////////////////////////////
// Dir (mandatory).  List of directory locations, e.g.
///////////////////////////////////////////////////////////
Dir
{
    //// Root (required): Specifies the path of the root of the FTP archive.
    Root "/srv/dak/ftp/";

    //// Pool (required): This variable specifies the path of the pool
    //// directory.  Debian packages will be placed in the pool by
    //// 'dak process-upload'
    Pool "/srv/dak/ftp/pool/";

    //// Templates (required): dak sends various mails and uses templates from
    //// this directory.
    Templates "/srv/dak/dak/templates/";

    //// Override (optional): This directory optionally stores override files (used
    //// by 'dak make-overrides').
    Override "/usr/local/dak/src/dak/scripts/override/";

    //// Lists (optional): This directory holds file lists used by apt-ftparchive to
    //// generate Packages and Sources files (used by 'dak make-suite-file-list').
    //// Deprecated in favour of use of generate-packages-sources2
    Lists "/srv/dak/database/dists/";

    //// Log (required): Directory to store dak logs in
    Log "/srv/dak/log/";

    //// Lock directory (required): Directory to store dak locks in
    Lock "/srv/dak/lock/";

    //// Morgue (required): Removed files are moved there.  The morgue has various
    //// sub-directories, including (optionally) those defined by
    //// Clean-Queues::MorgueSubDir and Clean-Suites::MorgueSubDir.
    Morgue "/srv/dak/morgue/";

    //// UrgencyLog (optional): If this directory is specified, 'dak
    //// process-upload' will store the urgency value of each upload.  This
    //// is mainly used for britney (the testing script).
    // UrgencyLog "/srv/dak/testing/urgencies/";

    //// Holding (required): Directory to use for temporary storage during
    //// process-upload
    Holding "/srv/dak/queue/holding/";

    //// Done (required): Directory in which to store processed .changes files
    Done "/srv/dak/queue/done/";

    //// BTSVersionTrack (optional): this directory holds the DebBugs
    //// Version Tracking support files.
    BTSVersionTrack "/srv/dak/btsversiontrack";

    //// Queue (required): This sub-tree defines important directories for the
    //// incoming queue.  The following variables have to be set: Byhand, New
    //// NewStage, Reject, Unchecked.  An explanation of the function of
    //// these directores can be found in README.new-incoming.
    Queue
    {
        Byhand "/srv/dak/queue/byhand/";
        New "/srv/dak/queue/new/";
        NewStage "/srv/dak/queue/newstage/";
        Reject "/srv/dak/queue/reject/";
        Unchecked "/srv/dak/queue/unchecked/";
    };
};

///////////////////////////////////////////////////////////
// SuiteMappings: (optional).  List of mappings for the Distribution file in a
// .changes file, e.g.:
///////////////////////////////////////////////////////////

// There are three mapping types:
//
// (1) map <source> <dest>
// Any target suite of '<source>' is unconditionally overriden to '<dest>'.
// (2) map-unreleased <source> <dest>
// Any upload targeted for suite '<source>' will be mapped to '<dest>' iff it
// contains uploads for an architecture that is not part of '<source>'.
// (3) ignore <suite>
// Any target suite of '<suite>' is unconditionally removed from the list of
// target suites.  NB: if the upload had only one target suite this will lead
// to rejection.

SuiteMappings
{
   "map stable proposed-updates";
   "map frozen unstable";
   "map-unreleased stable unstable";
   "map-unreleased proposed-updates unstable";
   "ignore testing";
};

///////////////////////////////////////////////////////////
// Urgency (mandatory) This defines the valid and default urgency of an upload.
// If a package is uploaded with an urgency not listed here, it will be
// rejected.
///////////////////////////////////////////////////////////

Urgency
{
  Default "low";
  Valid
  {
    low;
    medium;
    high;
    emergency;
    critical;
  };
};

--------------- /etc/dak/dak.conf ends --------------------

Any help or advice welcome - I really don't know what I'm doing here. However,
I'm able to proceed past this problem - separate mail for later problems.

Cheers

Simon



Please consider the environment before printing this email.


E-mail messages may contain viruses, worms, or other malicious code. By reading the message and opening any attachments, the recipient accepts full responsibility for taking protective action against such code. Henry Schein is not liable for any loss or damage arising from this message.

The information in this email is confidential and may be legally privileged. It is intended solely for the addressee(s). Access to this e-mail by anyone else is unauthorized.
Reply to:
Prev by Date: [dak/master] Don't be verbose in unchecked
Next by Date: DAK, Debian versions, and SQL Alchemy versions
Previous by thread: [dak/master] Don't be verbose in unchecked
Next by thread: DAK, Debian versions, and SQL Alchemy versions
Index(es):
- Date
- Thread