Re: [Emdebian] Mirrors
Let's discuss this with debian-mirrors folks. They have good experience
on setup mirrors for all kind of Debian related repository mirrors.
Hector has setup a push trigger script on emdebian.org, based on the
signal script example from http://www.debian.org/mirror/push_server
And the script works to trigger ftp.tw.d.o to kick up ftpsync script on
ftp.tw.d.o to sync from emdebian.org.
Question:
* What's the proper way to hook to send trigger to downstream mirrors on
emdebian.org server?
* What's the best path name to provide via rsyncd for mirrors?
Current repositories name available on emdebian.org are:
$ rsync -av emdebian.org::
debian emdebian tools and native packages archive
crush emdebian crush archive
grip emdebian grip archive
Folder name 'debian' is conflicted to main Debian's repo. So I placed
emdebian.org as:
http://ftp.tw.debian.org/emdebian.org/debian/
http://ftp.tw.debian.org/emdebian.org/crush/
http://ftp.tw.debian.org/emdebian.org/grip/
-Andrew
Hector Oron wrote:
> Hello,
>
> 2009/9/17 Neil Williams <codehelp@debian.org>:
>>> emdebian@ant:~/bin$ ./signal ftp.tw.debian.org push-mirror
>>> Signalling ftp.tw.debian.org
>>> ftp.tw.debian.org is unable to start rsync, lock file exists
>>>
>>> rsync was already running, so it was locked.
>> rsynd is running, rsync is not.
>
> Sure. I think you misunderstood me, ./signal is a <1sec task which
> just tells ftp.tw.debian.org to update the repository using rsync
> (above ftp.tw.debian.org was already running rsync that is why it says
> it is locked, but this as the test we did to proof that push_mirror
> was working.
>
>>> Do you mind to run (with user emdebian) the /home/emdebian/bin/signal
>>> script everytime you add new packages to the repository?
>> Not practical. With Grip, the packages are added by the
>> emdebian-grip-server package. The only real way is to do the call at
>> the end. Right now, grip-cron.sh appears to need more than several
>> hours to run - about 7-10hrs at the moment. I think it had something to
>> do with the outage because it usually takes quite a bit less and the
>> scripts are still trying to catch up with unstable. It could also be
>> simply dealing with much, much larger packages which take a lot of time
>> to Grip. I'm hoping to take a look at what is happening tomorrow but the
>> time required means that I can't actually do anything with the
>> repository in the evenings as reprepro is almost constantly locked (for
>> additions/updates).
>
> About your lags, i have been building toolchains on ant, that could
> explain your timings.
> According to Simon, the machine was down because KVM did not attached
> to the right network interface, but else it is posible too.
>
>>> Note that an scalable solution is better, because for each mirror we
>>> add, we need to add a signal entry
>> This kind of signal is not something that other machines running
>> emdebian-grip-server could support. Plus, adding a signal every time is
>> only going to turn grip-cron.sh into a 24hr script at which point it
>> becomes unusable.
>
> I am not aware (yet) how emdebian-grip-server and grip-cron.sh works,
> but the ./signal it is just a trigger.
> Anyone from emdebian server can talk to ftp.tw.debian.org and trigger
> the update by running such script in not much time, then the work is
> done by tw machine. :-)
>
> If you still think this model does not fit, i'll just set a cron task
> which triggers the update.
>
> BTW, is it OK to rename /org/emdebian/emdebian (actual crush -frozen-
> repository) to /org/emdebian/crush? I assume there are some /var/www
> symlinks to move arround, but is that all?
>
> Thanks :-)
>
Reply to: