Hi Olly,
Olly Betts <olly@survex.com> (2018-01-26):
> Yes. The directory on pkgmirror-csail actually had two different
> versions of the database using different database backends, so the
> files which aren't in the directory on picconi are essentially an
> old version of the same database.
>
> That alone shouldn't cause the search to fail though - it should just
> open one or the other (it looks like both Xapian 1.2.x and 1.4.x will
> open the old one in this rather odd situation).
>
> Xapian's replication should cleanly handle the database being replicated
> switching backends so I'm guessing the replication here is using rsync
> (without --delete-after) or similar? That might also explain why the
> old database is broken as the databases aren't safe to rsync if an
> update is in progress - if the last rsync of the old database happened
> while an update was in progress, it could have been left broken, which
> would typically result in particular searches failing.
>
> Updating with rsync can also break searches on the mirror while the
> rsync is in progress even if the master isn't updating, so perhaps we
> should switch to using Xapian's replication? That's already being
> used for the main website search (I set it up with weasel at Debconf
> 15).
>
> Happy to assist with that, though I'll probably need ssh access to
> pkgmirror-csail (seems I currently only have it for picconi). My
> Debian username is "olly".
Looking around without knowing much about pkg stuff:
- picconi has: conf/pushpdo.mirror → csail pkgmirror-csail.debian.org pkg_user
- those bits are read by bin/pushpdo, which calls rsync this way:
rsync -e "ssh -i ${MKEYFILE} -${MPROTO} ${SSH_OPTS}" -av --stats "${MIRRORPATH}" ${MUSER}@${MHOSTNAME}:/does/not/matter >"${LOGDIR}/pushpdo.${MLNAME}.log"
I suppose help is welcome to possibly fix/improve that, but I'm no pkg
expert (I just happened to have helped one or twice in the past, and to
still be in the pkg_maint group…)
Cheers,
--
Cyril Brulebois (kibi@debian.org) <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant
Attachment:
signature.asc
Description: PGP signature