[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: new buildds and devel alphas (Was: Advancing Debian Alpha Porting)

On 05-18 02:47, Adrian Zaugg wrote:
> Dear all
> On 14/04/11 10:48, Michael Cree wrote:
> > It appears we are to provide the Alpha buildds.  We need at least two
> I just was thinking abaout activating my ES-45 as a part of the job.
> Some questions arise, though.
> As a willing new buildd admin, I need to know about the expected data
> traffic volume, diskspace needs, how to sign built packages, how upload
> access is done. Could anyone of you please shortly outline? (Sid and
> experimental only? DD needed for signatures and upload?)
> This helps the people who have expressed their will to provide a buildd.
> Maybe it would be wise to summarize the planned new alpha port machines
> somewhere in the wiki (?), where the new admins can add their machines,
> if they still want to. I vaguely remember that every now and then there
> were also no alpha developer machines available - some machines could
> also serve as such, and be added there (with some Infos on the host,
> network, location, DD status and intended purpose maybe).

DSA already have such information for all debian.org machines,
and other ones. They are all publiclly availble on WWW somewhere.
(It is not a wiki, but just a webfronted for displaying data which is
already in LDAP database AFAIK). And it is read-only for security
reasons, as there are also things like SSH keys.

> And instead of everyone fighting to setup a buildd on their own, we
> could also share our experience and update the docu of this process,
> since there are several documents around, but none that seems really
> perfect. I found:
> - http://www.debian.org/devel/buildd/

> - http://dsa.debian.org/howto/new-machine/
> - https://buildd.debian.org/docs/buildd-setup.txt

I essentially used this two. I think at least 20GB dedicated for the buildd
on machine will be needed for each distribution (20 for testing, 20 for unstable,
20 for experimental). For sure best is to use LVM, as it really simplifies
setup, upgrades and cleanup (well it just clones logical volumes
without library or chroot magic).

Still this tutorials are somehow outdated, and was some time ago
asking for more uptodate official documentation of debian-ports.org.

I have some statistics of BIG packages:

#   linux-2.6: 3.9 GB, 12 h
#   libreoffice: 7.5 GB, 18 h
#   qt4-x11: 7.4 GB, 9 h
#   iceweasel: 3.2 GB, 2 h
#   gdc-4.4: 1.4 GB, 3 h
#   gcc-4.5: 2.1 GB, 11 h
#   llvm: 1 GB, 2 h (without tests)
#   eglibc: 1.2 GB, 4 h
#   ghc 7.0: 4.7 GB, 20 h (without tests)
#   eclipse: 3 GB, 11 h
#   gcc-4.6: 5.0+ GB

(times was taken from one of the alpha buildd in debian or other arch,
on my XP1000 it would be probably needed to multiply this times time 4).

Essentially no single package should take more than 8GB of disk space. Ever.
So somehow this 20GB is very safe value. I currently have 36GB in each machine
and would like to use them both for unstable probably, so it should
not be a problem.

The biggest problem for me is that buildd user needs essentially full
root access (via sudo). I currently was trying to prepare somehow
more restricted sudoers, but in essence I also needed to include /bin/bash,
/bin/mv and /bin/chmod, in list of allowed programs, which is pretty pointless
(Other was  /sbin/lvcreate, /sbin/mkfs.ext3, /bin/mount, /bin/umount,
/sbin/lvremove, /usr/sbin/debootstrap, /bin/mkdir, /bin/mv, /bin/chown, /bin/chmod)

My current setup is like this:


$ apt-get install sudo
$ apt-get install apt-transport-https

$ echo "deb https://buildd.debian.org/apt/ lenny main" >> /etc/apt/sources.list
$ wget -O buildd-apt.archive-key.asc https://buildd.debian.org/apt/archive-key.asc
$ apt-key add buildd-apt.archive-key.asc

$ apt-get install buildd-builder-meta
$ echo "debian_mirror=http://ftp.debian.semp.shell.la/debian/"; > /etc/schroot/conf.buildd    # my ftp.pl.debian.org mirror, needs changing now to ftp.X.debian-ports.org

# ustawienie adresow email w /etc/sbuild/sbuild.conf

$ wget http://cdn.debian.net/debian/pool/main/d/debootstrap/debootstrap_1.0.26+squeeze1_all.deb
$ dpkg -i ./debootstrap_1.0.26+squeeze1_all.deb     # because machine is lenny, but we need pretty recent debootstrap for squeezy and wheezy

$ adduser buildd
$ sbuild-adduser buildd

$ lvcreate --size 15G --name buildd-trees a09_array1_vg
$ install --directory --mode=2700 --owner=buildd --group=buildd ~buildd/build-trees
$ mkfs.ext3 /dev/a09_array1_vg/buildd-trees
$ echo "/dev/a09_array1_vg/buildd-trees /home/buildd/build-trees ext3 rw 0 2" >> /etc/fstab    # UUID may be better
$ mount /home/buildd/build-trees
$ chown -R buildd:buildd /home/buildd/build-trees
$ chmod 2700 /home/buildd/build-trees

$ echo "buildd  ALL=NOPASSWD: /sbin/lvcreate, /sbin/mkfs.ext3, /bin/mount, /bin/umount, /sbin/lvremove, /usr/sbin/debootstrap, /bin/mkdir, /bin/mv, /bin/chown, /bin/chmod, /bin/bash" >> /etc/sudoers
$ su - buildd
buildd$ cd /home/buildd
buildd$ chmod 2775 /home/buildd
buildd$ install --directory --mode=2770 --owner=buildd --group=buildd build logs old-logs upload-security
buildd$ install --directory --mode=2775 --owner=buildd --group=buildd stats stats/graphs upload
buildd$ echo '|/usr/bin/buildd-mail' > ~buildd/.forward

buildd$ for d in testing unstable; do /usr/share/sbuild/create-chroot $d a09_array1_vg 5G; done
# I'm not sure if 5GB is enough here. It is enough for debootrstrap, but what
# with space for the build-deps (this can be even around 2GB) and build (this can be around 5-10GB)?


> - http://www.debian-ports.org/archive
> - http://kmuto.jp/open.cgi?buildd
> Have I missed the one and only?
> Regards, Adrian.
> -- 
> To UNSUBSCRIBE, email to debian-alpha-REQUEST@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
> Archive: [🔎] 4DD3170A.50404@ente.limmat.ch">http://lists.debian.org/[🔎] 4DD3170A.50404@ente.limmat.ch

Witold Baryluk
JID: witold.baryluk // jabster.pl

Attachment: signature.asc
Description: Digital signature

Reply to: