dinstall/debian-installer
On Mon, Dec 09, 2002 at 10:39:43PM +0000, Roger Leigh wrote:
> There is some documentation for using it, but is there any for the
> initial setup?
>
> I certainly have read the docs from CVS, but I'm stuck at the
> installation stage. I can see what installs where from the debian/*
> packaging, but what I need to do to setup the archive and database
> isn't clear.
I have got it going!
It is pretty straight forward once you get to know what all these
names do. Even now though I still forget the correct name for the
correct purpose.
What I did:
- Edit katie.conf to suit. Make sure the path to the templates directory
is correct.
- Edit vars.
- Edit apt.conf so it has the same information as kate.conf. If there
are areas in apt.conf but not in katie.conf, you will get errors
when generating index files.
One day in the future I would like to see vars and apt.conf
autogenereted from katie.conf; I am not sure if this is
possible or not.
- Change paths to config files in utils.py.
- Configure postgresql to allow access and ability to create database.
- psql -f init_pool.sql template1
- alyson imports katie.conf into database.
- rose creates initial directories (actually some where missing; I can't
remember which onces were missing and which ones were misconfigured
though now).
- cron.unchecked checks files in queue/unchecked and moves them to
queue/new or queue/accepted.
- To manually accept queue/new or queue/byhand files, cd to that
directory, and type "lisa *.changes".
- Anything else I forgot to include.
- melanie marks packages for deletion. This doesn't happen straight
away, by default there is a delay (in katie.conf) before files are
actually removed.
(melanie complains that it cannot delete the files from buildd/*,
but I can't see why, I deleted the same files mentioned manually
without any problems).
- look at database with pgaccess often to try and find out what is
happening.
These files need customization:
- cron.daily installs accepted files and rebuilds the database. (do not
use udpate-* for private archives). Check apt-ftparchive doesn't
produce any errors, this might be because you have areas defined in
apt.conf but not in katie.conf.
- cron.buildd is for buildd, I don't know where to get wanna-build from
so I don't use it.
- cron.hourly doesn't appear to be required.
- cron.weekly might be important, but I don't understand what it does.
- cron.monthly doesn't appear to be required.
- mkmaintainers adds NON-US maintainers and base maintainers.
Potentially dangerous features:
- turn off BXA notifications in katie.conf. Or change the template.
- turn off auto closing bugs in katie.conf. Or at least change
the template.
- when new package is accepted it will inform uploader and package
maintainer; the maintainer might confuse this E-Mail as an
unauthorised upload to the Debian archive. I am not sure what the best
solution is here.
- some templates in templates/* have E-Mail addresses hardcoded (From
and To E-Mail addresses).
- cron.* have E-Mail addresses hardcoded.
Features I really dislike:
- cron.daily has:
kelly -pa *.changes | tee REPORT | \ mail -s "Install for $(date +%D)"
bam@snoopy.apana.org.au
This comes up with lots of errors if *.changes doesn't exist.
- No way to override component eg. (main,contrib,non-free) without
changing source; I have a patch that will let you override it in lisa.
I have also patched/hacked jennifer to use existing overide component
if one already exists.
This patch has the side effect that you can only have one version
of a package in all components combined; I don't see this as being
a problem.
See patch attached. It seems to work fine, but I haven't tested
it since replacing several tabs with spaces...
It might be better if the changes to jennifer were made optional
by a config switch.
lisa already has the ability to specify the component in the section,
but this didn't seem to be used.
- if you don't have testing, some of the scripts will fail. Testing is
hardcoded and required.
- *NO* E-Mail addresses should get hardcoded in non-config files
anywhere!
- cron.daily is pretty verbose.
- cron.daily generates override.potato.all3 and override.sid.all3, but I
can't see what use these files are.
What I don't like but can't really be fixed:
- GPG key for signing Release files can't be password protected, it
needs to work in a Cron job.
What I do like:
- good checking of packages, eg. to ensure that full source code could
be found. Then again a large number of my packages failed to install
because I forgot to use -sa when compiling :-(.
Still to learn:
- How to move packages to stable.
- How to add/remove distributions/components. It appears that you must
modify three things, 1. katie.conf, 2. apt.conf, 3. database. I really
don't like this, it is prone to error. alyson wont update a database
that isn't blank.
- How to best layout a private archive. At the moment I am thinking of
doing something similar to Debian, and initially uploading files to
unstable before I test them, and then move them to stable after I am
sure it works. This way other people can try out my packages even
though known bugs might exist.
I probably will not use testing, ever. I can't really see a good
reason for it, and there is no way a script could even tell
automatically if the version is buggy or not (no - I do not want to
setup my own BTS ;-) ).
- My idea all along was to create three components, one main, one ACL
(for files required to get access control lists going) and one for
SE-Linux (should be obvious). This means that if you don't want
SE-Linux files, you don't have to get them.
However, debian-installer will only check that main component for
udeb files.
I am not sure if I should just abandon the other two components and
put everything in main anyway, or what.
- Not sure how to move files between components, this might require a
delete operation followed by a new upload.
- Where is the check to ensure files aren't uploaded to stable?
- nothing except mkmaintainers seems to use the ftpdir/indices
directory???
If anybody wants any patches and/or config files, please let me know.
Hope this helps somebody.
--
Brian May <bam@debian.org>
Index: katie.py
===================================================================
RCS file: /cvs/dak/dak/katie.py,v
retrieving revision 1.27
diff -u -r1.27 katie.py
--- katie.py 16 Oct 2002 02:47:32 -0000 1.27
+++ katie.py 10 Dec 2002 23:39:34 -0000
@@ -668,6 +668,42 @@
################################################################################
+ def get_override_component (self, package, suite, binary_type, file):
+ files = self.pkg.files;
+
+ if binary_type == "": # must be source
+ type = "dsc";
+ else:
+ type = binary_type;
+
+ # Override suite name; used for example with proposed-updates
+ if self.Cnf.Find("Suite::%s::OverrideSuite" % (suite)) != "":
+ suite = self.Cnf["Suite::%s::OverrideSuite" % (suite)];
+
+ # Avoid <undef> on unknown distributions
+ suite_id = db_access.get_suite_id(suite);
+ if suite_id == -1:
+ return None;
+ type_id = db_access.get_override_type_id(type);
+
+ q = self.projectB.query("SELECT c.name FROM override o, component c WHERE package = '%s' AND suite = %s AND type = %s AND o.component = c.id"
+ % (package, suite_id, type_id));
+ result = q.getresult();
+ # If checking for a source package fall back on the binary override type
+ if type == "dsc" and not result:
+ deb_type_id = db_access.get_override_type_id("deb");
+ udeb_type_id = db_access.get_override_type_id("udeb");
+ q = self.projectB.query("SELECT c.name FROM override o, component c WHERE package = '%s' AND suite = %s AND (type = %s OR type = %s) AND o.component = c.id"
+ % (package, suite_id, deb_type_id, udeb_type_id));
+ result = q.getresult();
+
+ if result:
+ return result[0][0];
+ else:
+ return None;
+
+ ################################################################################
+
def reject (self, str, prefix="Rejected: "):
if str:
# Unlike other rejects we add new lines first to avoid trailing
Index: lisa
===================================================================
RCS file: /cvs/dak/dak/lisa,v
retrieving revision 1.20
diff -u -r1.20 lisa
--- lisa 26 Nov 2002 16:15:56 -0000 1.20
+++ lisa 10 Dec 2002 23:39:43 -0000
@@ -51,6 +51,7 @@
Logger = None;
Priorities = None;
+Components = None;
Sections = None;
reject_message = "";
@@ -271,6 +272,27 @@
################################################################################
+class Component_Completer:
+ def __init__ (self):
+ self.components = [];
+ q = projectB.query("SELECT name FROM component");
+ for i in q.getresult():
+ self.components.append(i[0]);
+
+ def complete(self, text, state):
+ if state == 0:
+ self.matches = [];
+ n = len(text);
+ for word in self.components:
+ if word[:n] == text:
+ self.matches.append(word);
+ try:
+ return self.matches[state]
+ except IndexError:
+ return None
+
+############################################################
+
class Section_Completer:
def __init__ (self):
self.sections = [];
@@ -315,9 +337,11 @@
def check_valid (new):
for pkg in new.keys():
+ component = new[pkg]["component"];
section = new[pkg]["section"];
priority = new[pkg]["priority"];
type = new[pkg]["type"];
+ new[pkg]["component id"] = db_access.get_component_id(component);
new[pkg]["section id"] = db_access.get_section_id(section);
new[pkg]["priority id"] = db_access.get_priority_id(new[pkg]["priority"]);
# Sanity checks
@@ -338,6 +362,10 @@
index += 1;
section = new[pkg]["section"];
priority = new[pkg]["priority"];
+ component = new[pkg]["component"];
+ if new[pkg]["component id"] == -1:
+ component += "[!]";
+ broken = 1;
if new[pkg]["section id"] == -1:
section += "[!]";
broken = 1;
@@ -345,9 +373,9 @@
priority += "[!]";
broken = 1;
if indexed:
- line = "(%s): %-20s %-20s %-20s" % (index, pkg, priority, section);
+ line = "(%s): %-20s %-20s %-20s %-20s" % (index, pkg, priority, component, section);
else:
- line = "%-20s %-20s %-20s" % (pkg, priority, section);
+ line = "%-20s %-20s %-20s %-20s" % (pkg, priority, component, section);
line = line.strip()+'\n';
file.write(line);
note = Katie.pkg.changes.get("lisa note");
@@ -411,19 +439,23 @@
continue;
s = line.split();
# Pad the list if necessary
- s[len(s):3] = [None] * (3-len(s));
- (pkg, priority, section) = s[:3];
+ s[len(s):4] = [None] * (4-len(s));
+ (pkg, priority, component, section) = s[:4];
if not new.has_key(pkg):
utils.warn("Ignoring unknown package '%s'" % (pkg));
else:
# Strip off any invalid markers, print_new will readd them.
+ if component.endswith("[!]"):
+ component = component[:-3];
if section.endswith("[!]"):
section = section[:-3];
if priority.endswith("[!]"):
priority = priority[:-3];
for file in new[pkg]["files"]:
+ Katie.pkg.files[file]["component"] = component;
Katie.pkg.files[file]["section"] = section;
Katie.pkg.files[file]["priority"] = priority;
+ new[pkg]["component"] = component;
new[pkg]["section"] = section;
new[pkg]["priority"] = priority;
@@ -431,6 +463,7 @@
def edit_index (new, index):
priority = new[index]["priority"]
+ component = new[index]["component"]
section = new[index]["section"]
type = new[index]["type"];
done = 0
@@ -439,10 +472,10 @@
answer = "XXX";
if type != "dsc":
- prompt = "[B]oth, Priority, Section, Done ? ";
+ prompt = "[B]oth, Priority, Component, Section, Done ? ";
else:
prompt = "[S]ection, Done ? ";
- edit_priority = edit_section = 0;
+ edit_priority = edit_component = edit_section = 0;
while prompt.find(answer) == -1:
answer = utils.our_raw_input(prompt);
@@ -453,6 +486,8 @@
if answer == 'P':
edit_priority = 1;
+ elif answer == 'C':
+ edit_component = 1;
elif answer == 'S':
edit_section = 1;
elif answer == 'B':
@@ -472,6 +507,17 @@
got_priority = 1;
priority = new_priority;
+ if edit_component:
+ readline.set_completer(Components.complete);
+ got_component = 0;
+ while not got_component:
+ new_component = utils.our_raw_input("New component: ").strip();
+ if Components.components.count(new_component) == 0:
+ print "E: '%s' is not a valid component, try again." % (new_component);
+ else:
+ got_component = 1;
+ component = new_component;
+
# Edit the section
if edit_section:
readline.set_completer(Sections.complete);
@@ -489,8 +535,10 @@
for file in new[index]["files"]:
Katie.pkg.files[file]["section"] = section;
+ Katie.pkg.files[file]["component"] = component;
Katie.pkg.files[file]["priority"] = priority;
new[index]["priority"] = priority;
+ new[index]["component"] = component;
new[index]["section"] = section;
return new;
@@ -507,7 +555,7 @@
index += 1;
new_index[index] = i;
- prompt = "(%s) edit override <n>, Editor, Done ? " % (index_range(index));
+ prompt = "(%s) edit override <n>, Component change for all, Editor, Done ? " % (index_range(index));
got_answer = 0
while not got_answer:
@@ -515,6 +563,21 @@
answer = answer[:1].upper();
if answer == "E" or answer == "D":
got_answer = 1;
+
+ elif answer == "C":
+ readline.set_completer(Components.complete);
+ got_component = 0;
+ while not got_component:
+ new_component = utils.our_raw_input("New component: ").strip();
+ if Components.components.count(new_component) == 0:
+ print "E: '%s' is not a valid component, try again." % (new_component);
+ else:
+ got_component = 1;
+ for pkg in new.keys():
+ new[pkg]["component"] = new_component;
+ for file in new[pkg]["files"]:
+ Katie.pkg.files[file]["component"] = new_component;
+
elif katie.re_isanum.match (answer):
answer = int(answer);
if (answer < 1) or (answer > index):
@@ -622,9 +685,9 @@
for suite in changes["suite"].keys():
suite_id = db_access.get_suite_id(suite);
for pkg in new.keys():
- component_id = db_access.get_component_id(new[pkg]["component"]);
type_id = db_access.get_override_type_id(new[pkg]["type"]);
priority_id = new[pkg]["priority id"];
+ component_id = new[pkg]["component id"];
section_id = new[pkg]["section id"];
projectB.query("INSERT INTO override (suite, component, type, package, priority, section) VALUES (%s, %s, %s, '%s', %s, %s)" % (suite_id, component_id, type_id, pkg, priority_id, section_id));
for file in new[pkg]["files"]:
@@ -730,7 +793,7 @@
################################################################################
def init():
- global Cnf, Options, Logger, Katie, projectB, Sections, Priorities;
+ global Cnf, Options, Logger, Katie, projectB, Components, Sections, Priorities;
Cnf = utils.get_conf();
@@ -761,6 +824,7 @@
projectB = Katie.projectB;
+ Components = Component_Completer();
Sections = Section_Completer();
Priorities = Priority_Completer();
readline.parse_and_bind("tab: complete");
Index: jennifer
===================================================================
RCS file: /cvs/dak/dak/jennifer,v
retrieving revision 1.28
diff -u -r1.28 jennifer
--- jennifer 21 Oct 2002 13:54:53 -0000 1.28
+++ jennifer 10 Dec 2002 23:39:52 -0000
@@ -663,6 +663,12 @@
reject("unknown component `%s' for suite `%s'." % (files[file]["component"], suite));
continue
+ #hack to use existing override component, if one exists
+ override_component = Katie.get_override_component(files[file]["package"], suite, files[file].get("dbtype",""), file);
+
+ if override_component:
+ files[file]["component"] = override_component;
+
# See if the package is NEW
if not Katie.in_override_p(files[file]["package"], files[file]["component"], suite, files[file].get("dbtype",""), file):
files[file]["new"] = 1;
@@ -1137,7 +1143,7 @@
raise;
except:
print "ERROR";
- traceback.print_exc(file=sys.stderr);
+ traceback.print_exc(file=sys.stderr);
pass;
# Restore previous WD
Reply to: