On Fri, 18 Sep 2009 15:54:58 +0800 Andrew Lee <ajqlee@debian.org> wrote: > Let's discuss this with debian-mirrors folks. They have good > experience on setup mirrors for all kind of Debian related repository > mirrors. True - however we don't quite do things as Debian would do things, principally due to the lack of an incoming.emdebian.org staging area that results in the archive being continually updated. This is due to a technical limitation (lack of sufficient space) and changes in the scripts to cope with the extra space. debian-mirrors people: feel free to ignore the thread until such time as Grip has a more compatible setup. Sorry for any noise. > Hector has setup a push trigger script on emdebian.org, based on the > signal script example from http://www.debian.org/mirror/push_server > > And the script works to trigger ftp.tw.d.o to kick up ftpsync script > on ftp.tw.d.o to sync from emdebian.org. > > Question: > * What's the proper way to hook to send trigger to downstream mirrors > on emdebian.org server? Get more space onto www.emdebian.org so that we can use a more Debian-like process in the first place at which point normal Debian mirror techniques become appropriate. Right now, mirroring only makes things worse. > * What's the best path name to provide via rsyncd for mirrors? > Current repositories name available on emdebian.org are: > $ rsync -av emdebian.org:: > debian emdebian tools and native packages archive > crush emdebian crush archive > grip emdebian grip archive > > Folder name 'debian' is conflicted to main Debian's repo. So I placed > emdebian.org as: > http://ftp.tw.debian.org/emdebian.org/debian/ > http://ftp.tw.debian.org/emdebian.org/crush/ > http://ftp.tw.debian.org/emdebian.org/grip/ IIRC the toolchains will be migrating into Debian anyway so the whole debian/ repository could become moot. The native package support in debian/ is also largely unused now - principally because the packages themselves do not need such rapid changes that we need a new version before the old one has migrated into testing. That should leave only crush/ and grip/ However, I repeat my request that people *don't* mirror Grip but instead run partial buildd's using the current emdebian-grip-server package to add more packages to Grip. We don't need mirrors right now, we need more machines preparing more packages resulting in bug reports against emdebian-grip-server and hence better code in the package itself. Also, moving Grip processing to a backend server would free up space for more packages to go onto the main emdebian.org server - as long as that server isn't the one doing the work. This improvement has little to do with mirroring. Until we have an incoming staging area, I don't think it's practical to mirror Grip or involve the debian-mirrors people. > -Andrew (please don't top post) > >> Not practical. With Grip, the packages are added by the > >> emdebian-grip-server package. The only real way is to do the call > >> at the end. Right now, grip-cron.sh appears to need more than > >> several hours to run - about 7-10hrs at the moment. I think it had > >> something to do with the outage because it usually takes quite a > >> bit less and the scripts are still trying to catch up with > >> unstable. It could also be simply dealing with much, much larger > >> packages which take a lot of time to Grip. I'm hoping to take a > >> look at what is happening tomorrow but the time required means > >> that I can't actually do anything with the repository in the > >> evenings as reprepro is almost constantly locked (for > >> additions/updates). > > > > About your lags, i have been building toolchains on ant, that could > > explain your timings. > > According to Simon, the machine was down because KVM did not > > attached to the right network interface, but else it is posible too. > > > >>> Note that an scalable solution is better, because for each mirror > >>> we add, we need to add a signal entry > >> This kind of signal is not something that other machines running > >> emdebian-grip-server could support. Plus, adding a signal every > >> time is only going to turn grip-cron.sh into a 24hr script at > >> which point it becomes unusable. See: http://lists.debian.org/debian-embedded/2009/09/msg00039.html -- Neil Williams ============= http://www.data-freedom.org/ http://www.linux.codehelp.co.uk/ http://e-mail.is-not-s.ms/
Attachment:
pgppTJ2QbCulg.pgp
Description: PGP signature