(please CC on replies)
Hi,
We'd like to get some advise on the plans to move the KGB service
provided for alioth/salsa-hosted projects and others, to
a debian-hosted machine.
https://wiki.debian.org/Services/KGB
Current architecture
====================
Three IRC bots, running on home servers. Client part is chained in
post-commit hooks, running on the Git[1] server (alioth and others).
[1] the client also works with Subversion and CVS (used by the web
team)
When a push/commit/checkin is made, the client processes the change
and sends one of the bots a notification to be relayed on IRC. If
there is a problem with a given bot, it retries with another, until
the list is over, at which point a warning is issued.
Each repository needs to be configured on the bot side, so that
requests can be authenticated. Every repository is given access to
a fixed number of IRC channels. All requests are authenticated using
a hash over the request content and a pre-shared secret.
Apart from post-commit hooks, there is a separate functionality which
allows sending free-text messages to IRC, suitable for purposes other
than commit notification - bug submission, monitoring alert, job
completion etc.
This approach has two downsides:
- every new project needs to be configured by bot admins (e.g. three
times)
- sometimes bots are inaccessible (e.g. power outage or hardware/link
problems). This is somewhat mitigated by the client retrying with
a different bot, but there is still a delay.
The future
==========
Most of the projects move to salsa, which doesn't support post-commit
hooks, but can use webhooks.
For these, a webhook termination is implemented in the bot directly.
The webhook termination code doesn't need authentication, because it
trusts salsa's IP address. The variable parts of the notification
(e.g. channel/network, use of colors etc) can be embedded in the
webhook URI. We trust the projects not to abuse the system by
broadcasting advertisements to thousands of IRC channels (because they
obey DMUP, and there is a rate limit).
On service maintainers' side, this avoids the need to configure
individual salsa projects. For users it is also much easier to enable
IRC notifications - instead of fiddling with post-commit hooks and
their configuration they enable the webhook support in gitlab as
described in https://salsa.debian.org/kgb-team/kgb/wikis/usage
The "old" scheme with authenticated projects stays, because it would
be used by the non-gitlab projects and the projects that need
free-text notifications.
To address both the stability issue with home servers and the need to
configure several servers, we'd like to move the service to a single
debian-hosted machine.
Hosting requirements
====================
The expected load is rather low, CPU% is seldom higher than 0.0.
Memory footprint was seen to be 200M after 30 days of running, and the
expected network bandwidth is about xxx MB/month.
In terms of access, we would want to be able to:
- install/upgrade kgb-bot and kgb-client packages (plus
dependencies), possibly from backports or unstable (or from git)
- read/modify KGB's configuration:
- modify files under /etc/kgb-bot/*
- create/modify/delete files under /etc/kgb-bot/kgb.d/
currently permissions look like
drwxr-xr-x root root /etc/kgb-bot
-rw-r----- root Debian-kgb kgb.conf
drwxr-x--- root Debian-kgb kgb.conf.d
-rw-r----- root Debian-kgb kgb.conf.dpkg-dist
-rw-r----- root Debian-kgb 3dprinter.conf
-rw-r----- root Debian-kgb android-tools.conf
-rw-r----- root Debian-kgb apt.conf
-rw-r----- root Debian-kgb archlinux-reproducible.conf
-rw-r----- root Debian-kgb boinc.conf
.....
- read /var/log/kgb-bot.log* (-rw-r--r-- Debian-kgb:root)
For the KGB service admins,
Damyan Ivanov
Attachment:
signature.asc
Description: PGP signature