[RFH] Local wanna-build and buildd setup in unstable
Hello everybody,
I am trying to setup a local wanna-build instance together with a
local buildd and a local package repository.
I have read the general buildd-network documentation
(https://www.debian.org/devel/buildd/ and friends),
the HOWTO in the Debian wiki
(https://wiki.debian.org/DebianWannaBuildInfrastructureOnOneServer)
and several mails on the mailing lists regarding the topic.
Nonetheless I am still struggeling with the setup and could use
some help. Most older documentation states something like "if
things are unclear, just log in and take a look at the official
Debian wanna-build installation", but that is of course not
possible anymore as wanna-build and the buildds nowadays run on
restricted machines.
My setup is based on the wanna-build git repository at
https://buildd.debian.org/git/wanna-build.git/ and follows the
HOWTO in the Debian wiki (see above).
The first problem came up when generating the wanna-build SQL
database from the scripts in /srv/wanna-build/schema. When
running the /srv/wanna-build/schema/arches-tables.sh script with
all arches enabled and then running the arches-tables.sql script
created by it, postgresql chokes on architecture names containing
a dash (kfreebsd-amd64, kfreebsd-i386).
>From what I found in the postgresql documentation,
identifiers containing dashes seem to need to be quoted, which is
done only partially in /srv/wanna-build/schema/arches-tables.in.
I have tried to add quotes around the architecture names where
missing, resulting in the attached patch against wanna-build.
With this patch, the inital database setup appears to run
without unexpected errors, i.e. without errors besides those
listed at https://wiki.debian.org/DebianWannaBuildInfrastructureOnOneServer.
When running the trigger.local script (from
https://wiki.debian.org/DebianWannaBuildInfrastructureOnOneServer?action=AttachFile&do=view&target=trigger.local)
to feed the state of my local package repository into wanna-build,
it fails with
DBD::Pg::db selectall_hashref failed: FEHLER: Spalte „mail_logs“ existiert nicht
LINE 1: ..._dep_resolver, suppress_successful_logs, archive, mail_logs ...
^ at /srv/wanna-build/bin/wanna-build line 310.
DBD::Pg::db selectall_hashref failed: FEHLER: aktuelle Transaktion wurde
abgebrochen, Befehle werden bis zum Ende der Transaktion ignoriert at
/srv/wanna-build/bin/wanna-build line 322.
Bad distribution 'sid'
Wanna-build tries to access a column "mail_logs" in the table
"distributions" which is not found and which indeed does not get generated
by the wanna-build SQL scripts. Main-tables.sql defines the distributions
table as:
CREATE TABLE distributions (
distribution character varying NOT NULL,
public boolean DEFAULT true NOT NULL,
sort_order integer DEFAULT 0,
auto_dep_wait boolean DEFAULT true,
build_dep_resolver character varying,
archive character varying,
suppress_successful_logs boolean DEFAULT false NOT NULL
);
When looking into the file /srv/wanna-build/schema/wanna-build.sql,
which is not mentioned in the documentation and seems to be a dump
of the wanna-build database generated sometime in the past, one
finds a definition for the mail_logs column:
CREATE TABLE distributions (
distribution character varying NOT NULL,
public boolean DEFAULT true NOT NULL,
sort_order integer DEFAULT 0,
auto_dep_wait boolean DEFAULT true,
build_dep_resolver character varying,
archive character varying,
suppress_successful_logs boolean DEFAULT false NOT NULL,
mail_logs character varying
);
I have then added the missing column with
wannadb=> ALTER TABLE distributions ADD COLUMN mail_logs character varying;
With this, the trigger script runs successfully and running wanna-build
manually seems to work and gives results like
shells/csh_20110502-2: Needs-Build [optional:uncompiled:calprio{23}:days{0}]
which looks ok. When running buildd (0.64.1-1, current version in unstable),
it tries to access wanna-build via ssh, which fails seemingly due to the command
format not being understood by wanna-build:
Jan 26 20:50:18 buildd[5879]: D2: PROGRAM: /bin/sh
Jan 26 20:50:18 buildd[5879]: D2: COMMAND: ssh -l wbadm 127.0.0.1 wanna-build --arch=amd64 --user=wbadm --api 1 --dist=sid D2:
Jan 26 20:50:18 buildd[5879]: D2: INTCOMMAND: ssh -l wbadm 127.0.0.1 wanna-build --arch=amd64 --user=wbadm --api 1 --dist=sid D2:
Jan 26 20:50:18 buildd[5879]: D2: EXPCOMMAND: /bin/sh -c cd '/var/lib/buildd/build' && 'ssh' '-l' 'wbadm' '127.0.0.1' 'wanna-build' '--arch=amd64' '--user=wbadm' '--api 1' '--dist=sid' 'D2:'
Jan 26 20:50:18 buildd[5879]: D2: Environment set:
Jan 26 20:50:18 buildd[5879]: D: Running command: /bin/sh -c cd '/var/lib/buildd/build' && 'ssh' '-l' 'wbadm' '127.0.0.1' 'wanna-build' '--arch=amd64' '--user=wbadm' '--api 1' '--dist=sid' 'D2:'
Jan 26 20:50:18 buildd[5879]: Smartmatch is experimental at /usr/local/bin/wanna-build line 145.
Jan 26 20:50:18 buildd[5879]: given is experimental at /usr/local/bin/wanna-build line 156.
Jan 26 20:50:18 buildd[5879]: when is experimental at /usr/local/bin/wanna-build line 157.
Jan 26 20:50:18 buildd[5879]: when is experimental at /usr/local/bin/wanna-build line 161.
Jan 26 20:50:18 buildd[5879]: when is experimental at /usr/local/bin/wanna-build line 162.
Jan 26 20:50:18 buildd[5879]: when is experimental at /usr/local/bin/wanna-build line 163.
Jan 26 20:50:18 buildd[5879]: when is experimental at /usr/local/bin/wanna-build line 164.
Jan 26 20:50:18 buildd[5879]: defined(@array) is deprecated at /usr/local/bin/wanna-build line 412.
Jan 26 20:50:18 buildd[5879]: (Maybe you should just omit the defined()?)
Jan 26 20:50:18 buildd[5879]: D2:: can't extract package name and version (bad format)
Jan 26 20:50:18 buildd[5879]: Nothing to do -- sleeping 300 seconds
While there is documentation of the "normal" user commands to
wanna-build, I have found no documentation describing the API
used by buildd. I would appreciate very much if somebody could
point me into the right direction. I am unsure whether I am just
missing something obvious, whether the documentation is
incomplete or not up to date or whether there are bugs in
wanna-build and/or buildd.
Regards,
Karsten
--
Gem. Par. 28 Abs. 4 Bundesdatenschutzgesetz widerspreche ich der Nutzung
sowie der Weitergabe meiner personenbezogenen Daten für Zwecke der
Werbung sowie der Markt- oder Meinungsforschung.
diff --git a/schema/arches-tables.in b/schema/arches-tables.in
index a27a9e4..035977d 100644
--- a/schema/arches-tables.in
+++ b/schema/arches-tables.in
@@ -9,15 +9,15 @@ SET escape_string_warning = off;
-- Name: ARCH; Type: SCHEMA; Schema: -; Owner: wbadm
--
-DROP SCHEMA IF EXISTS ARCH;
-CREATE SCHEMA ARCH;
-ALTER SCHEMA ARCH OWNER TO wbadm;
+DROP SCHEMA IF EXISTS "ARCH";
+CREATE SCHEMA "ARCH";
+ALTER SCHEMA "ARCH" OWNER TO wbadm;
--
-- Name: ARCH_public; Type: SCHEMA; Schema: -; Owner: wbadm
--
-DROP SCHEMA IF EXISTS ARCH_public;
+DROP SCHEMA IF EXISTS "ARCH_public";
CREATE SCHEMA "ARCH_public";
ALTER SCHEMA "ARCH_public" OWNER TO wbadm;
@@ -25,37 +25,37 @@ ALTER SCHEMA "ARCH_public" OWNER TO wbadm;
-- Name: ARCH.packages; Type: VIEW; Schema: ARCH; Owner: wbadm
--
-DROP VIEW IF EXISTS ARCH.packages;
-CREATE VIEW ARCH.packages AS
+DROP VIEW IF EXISTS "ARCH".packages;
+CREATE VIEW "ARCH".packages AS
SELECT packages.package, packages.distribution, packages.version, packages.state, packages.section, packages.priority, packages.installed_version, packages.previous_state, packages.state_change, packages.notes, packages.builder, packages.failed, packages.old_failed, packages.binary_nmu_version, packages.binary_nmu_changelog, packages.failed_category, packages.permbuildpri, packages.buildpri, packages.depends, packages.rel, packages.bd_problem, packages.extra_depends, packages.extra_conflicts, packages.build_arch_all FROM public.packages packages WHERE ((packages.architecture)::text = 'ARCH'::text);
-ALTER TABLE ARCH.packages OWNER TO wbadm;
+ALTER TABLE "ARCH".packages OWNER TO wbadm;
--
-- Name: ARCH.pkg_history; Type: VIEW; Schema: ARCH; Owner: wbadm
--
-DROP VIEW IF EXISTS ARCH.pkg_history;
-CREATE VIEW ARCH.pkg_history AS
+DROP VIEW IF EXISTS "ARCH".pkg_history;
+CREATE VIEW "ARCH".pkg_history AS
SELECT pkg_history.package, pkg_history.distribution, pkg_history.version, pkg_history."timestamp", pkg_history.result, pkg_history.build_time, pkg_history.disk_space, pkg_history.builder FROM public.pkg_history WHERE ((pkg_history.architecture)::text = 'ARCH'::text);
-ALTER TABLE ARCH.pkg_history OWNER TO wbadm;
+ALTER TABLE "ARCH".pkg_history OWNER TO wbadm;
--
-- Name: ARCH.transactions; Type: VIEW; Schema: ARCH; Owner: wbadm
--
-DROP VIEW IF EXISTS ARCH.transactions;
-CREATE VIEW ARCH.transactions AS
+DROP VIEW IF EXISTS "ARCH".transactions;
+CREATE VIEW "ARCH".transactions AS
SELECT transactions.package, transactions.distribution, transactions.version, transactions.action, transactions.prevstate, transactions.state, transactions.real_user, transactions.set_user, transactions."time" FROM public.transactions WHERE ((transactions.architecture)::text = 'ARCH'::text);
-ALTER TABLE ARCH.transactions OWNER TO wbadm;
+ALTER TABLE "ARCH".transactions OWNER TO wbadm;
--
-- Name: ARCH.users; Type: VIEW; Schema: ARCH; Owner: wbadm
--
-DROP VIEW IF EXISTS ARCH.users;
-CREATE VIEW ARCH.users AS
+DROP VIEW IF EXISTS "ARCH".users;
+CREATE VIEW "ARCH".users AS
SELECT users.username, users.distribution, users.last_seen FROM public.users WHERE ((users.architecture)::text = 'ARCH'::text);
-ALTER TABLE ARCH.users OWNER TO wbadm;
+ALTER TABLE "ARCH".users OWNER TO wbadm;
--
-- Name: "ARCH_public".packages; Type: VIEW; Schema: ARCH_public; Owner: wbadm
@@ -93,7 +93,7 @@ CREATE VIEW "ARCH_public".users AS
SELECT users.username, users.distribution, users.last_seen FROM (public.users NATURAL JOIN public.distributions) WHERE ((distributions.public = true) AND ((users.architecture)::text = 'ARCH'::text));
ALTER TABLE "ARCH_public".users OWNER TO wbadm;
-SET search_path = ARCH, pg_catalog;
+SET search_path = "ARCH", pg_catalog;
--
-- Name: packages_delete; Type: RULE; Schema: ARCH; Owner: wbadm
@@ -169,67 +169,67 @@ SET search_path = public, pg_catalog;
-- Name: ARCH; Type: ACL; Schema: -; Owner: wbadm
--
-REVOKE ALL ON SCHEMA ARCH FROM PUBLIC;
-REVOKE ALL ON SCHEMA ARCH FROM wbadm;
-GRANT ALL ON SCHEMA ARCH TO wbadm;
-GRANT USAGE ON SCHEMA ARCH TO ARCH;
-GRANT USAGE ON SCHEMA ARCH TO wb_all;
-GRANT USAGE ON SCHEMA ARCH TO wb_security;
+REVOKE ALL ON SCHEMA "ARCH" FROM PUBLIC;
+REVOKE ALL ON SCHEMA "ARCH" FROM wbadm;
+GRANT ALL ON SCHEMA "ARCH" TO wbadm;
+GRANT USAGE ON SCHEMA "ARCH" TO "ARCH";
+GRANT USAGE ON SCHEMA "ARCH" TO wb_all;
+GRANT USAGE ON SCHEMA "ARCH" TO wb_security;
--
-- Name: ARCH_public; Type: ACL; Schema: -; Owner: wbadm
--
-REVOKE ALL ON SCHEMA ARCH_public FROM PUBLIC;
-REVOKE ALL ON SCHEMA ARCH_public FROM wbadm;
-GRANT ALL ON SCHEMA ARCH_public TO wbadm;
-GRANT USAGE ON SCHEMA ARCH_public TO ARCH;
-GRANT USAGE ON SCHEMA ARCH_public TO PUBLIC;
+REVOKE ALL ON SCHEMA "ARCH_public" FROM PUBLIC;
+REVOKE ALL ON SCHEMA "ARCH_public" FROM wbadm;
+GRANT ALL ON SCHEMA "ARCH_public" TO wbadm;
+GRANT USAGE ON SCHEMA "ARCH_public" TO "ARCH";
+GRANT USAGE ON SCHEMA "ARCH_public" TO PUBLIC;
--
-- Name: ARCH.packages; Type: ACL; Schema: ARCH; Owner: wbadm
--
-REVOKE ALL ON TABLE ARCH.packages FROM PUBLIC;
-REVOKE ALL ON TABLE ARCH.packages FROM wbadm;
-GRANT ALL ON TABLE ARCH.packages TO wbadm;
-GRANT SELECT ON TABLE ARCH.packages TO wb_security;
-GRANT SELECT,INSERT,UPDATE ON TABLE ARCH.packages TO wb_all;
+REVOKE ALL ON TABLE "ARCH".packages FROM PUBLIC;
+REVOKE ALL ON TABLE "ARCH".packages FROM wbadm;
+GRANT ALL ON TABLE "ARCH".packages TO wbadm;
+GRANT SELECT ON TABLE "ARCH".packages TO wb_security;
+GRANT SELECT,INSERT,UPDATE ON TABLE "ARCH".packages TO wb_all;
--
-- Name: ARCH.pkg_history; Type: ACL; Schema: ARCH; Owner: wbadm
--
-REVOKE ALL ON TABLE ARCH.pkg_history FROM PUBLIC;
-REVOKE ALL ON TABLE ARCH.pkg_history FROM wbadm;
-GRANT ALL ON TABLE ARCH.pkg_history TO wbadm;
-GRANT SELECT ON TABLE ARCH.pkg_history TO wb_security;
-GRANT SELECT,INSERT,UPDATE ON TABLE ARCH.pkg_history TO wb_all;
+REVOKE ALL ON TABLE "ARCH".pkg_history FROM PUBLIC;
+REVOKE ALL ON TABLE "ARCH".pkg_history FROM wbadm;
+GRANT ALL ON TABLE "ARCH".pkg_history TO wbadm;
+GRANT SELECT ON TABLE "ARCH".pkg_history TO wb_security;
+GRANT SELECT,INSERT,UPDATE ON TABLE "ARCH".pkg_history TO wb_all;
--
-- Name: ARCH.transactions; Type: ACL; Schema: ARCH; Owner: wbadm
--
-REVOKE ALL ON TABLE ARCH.transactions FROM PUBLIC;
-REVOKE ALL ON TABLE ARCH.transactions FROM wbadm;
-GRANT ALL ON TABLE ARCH.transactions TO wbadm;
-GRANT SELECT ON TABLE ARCH.transactions TO wb_security;
-GRANT SELECT,INSERT ON TABLE ARCH.transactions TO wb_all;
+REVOKE ALL ON TABLE "ARCH".transactions FROM PUBLIC;
+REVOKE ALL ON TABLE "ARCH".transactions FROM wbadm;
+GRANT ALL ON TABLE "ARCH".transactions TO wbadm;
+GRANT SELECT ON TABLE "ARCH".transactions TO wb_security;
+GRANT SELECT,INSERT ON TABLE "ARCH".transactions TO wb_all;
--
-- Name: ARCH.users; Type: ACL; Schema: ARCH; Owner: wbadm
--
-REVOKE ALL ON TABLE ARCH.users FROM PUBLIC;
-REVOKE ALL ON TABLE ARCH.users FROM wbadm;
-GRANT ALL ON TABLE ARCH.users TO wbadm;
-GRANT SELECT ON TABLE ARCH.users TO wb_security;
-GRANT SELECT,INSERT,UPDATE ON TABLE ARCH.users TO wb_all;
+REVOKE ALL ON TABLE "ARCH".users FROM PUBLIC;
+REVOKE ALL ON TABLE "ARCH".users FROM wbadm;
+GRANT ALL ON TABLE "ARCH".users TO wbadm;
+GRANT SELECT ON TABLE "ARCH".users TO wb_security;
+GRANT SELECT,INSERT,UPDATE ON TABLE "ARCH".users TO wb_all;
--
--
1.8.5.3
Reply to: