RE: rsync errors when using latest ftpsync

["Steve \(IWWD\)" <steve@iwwd.com>]

Hi Bastian,

Thank you for the reply.  

After testing further, it seems the issue was indeed related to the 'owner' permissions.  Once I had recursively set both the 'owner' and 'group' to the ftpsync executing user, the update ran correctly and links were created fine.

The problem it seems is that rsync, whenever it creates a new file, it's setting the owner as 'root', but is setting the group as the executing user.

Could there be an issue with the rsync command or something I'm missing?


This is the log from the last execution:

Oct 14 12:00:01 debian ftpsync[19935]: Mirrorsync start
Oct 14 12:00:01 debian ftpsync[19935]: Running mirrorsync, update is required, /mirrors/debian//Archive-Update-Required-debian.localdomain exists
Oct 14 12:00:01 debian ftpsync[19935]: Running stage1: rsync --filter=exclude_/Archive-Update-in-Progress-debian.localdomain --filter=protect_/Archive-Update-in-Progress-debian.localdomain --filter=exclude_/Archive-Update-Required-debian.localdomain --filter=protect_/Archive-Update-Required-debian.localdomain --filter=exclude_/project/trace/debian.localdomain --filter=protect_/project/trace/debian.localdomain --filter=exclude_/project/trace/debian.localdomain-stage1 --filter=protect_/project/trace/debian.localdomain-stage1 --filter=exclude_/project/trace/_hierarchy --filter=protect_/project/trace/_hierarchy --filter=exclude_/project/trace/_traces --filter=protect_/project/trace/_traces --filter=include_/project/ --filter=protect_/project/ --filter=include_/project/trace/ --filter=protect_/project/trace/ --filter=include_/project/trace/* --bwlimit=0 -prltvHSB8192 --timeout 3600 --stats --include=*.diff/ --exclude=*.diff/Index --exclude=Packages* --exclude=Sources* --exclude=Release* --exclude=InRelease --include=i18n/by-hash --exclude=i18n/* --exclude=ls-lR* --exclude=.~tmp~/ --exclude=/dists/**/binary-alpha/ --exclude=/dists/**/installer-alpha/ --exclude=/dists/**/Contents-alpha.gz --exclude=/dists/**/Contents-udeb-alpha.gz --exclude=/dists/**/Contents-alpha.diff/ --exclude=/indices/**/arch-alpha.files --exclude=/indices/**/arch-alpha.list.gz --exclude=/pool/**/*_alpha.deb --exclude=/pool/**/*_alpha.udeb --exclude=/pool/**/*_alpha.changes --exclude=/dists/**/binary-arm/ --exclude=/dists/**/installer-arm/ --exclude=/dists/**/Contents-arm.gz --exclude=/dists/**/Contents-udeb-arm.gz --exclude=/dists/**/Contents-arm.diff/ --exclude=/indices/**/arch-arm.files --exclude=/indices/**/arch-arm.list.gz --exclude=/pool/**/*_arm.deb --exclude=/pool/**/*_arm.udeb --exclude=/pool/**/*_arm.changes --exclude=/dists/**/binary-armhf/ --exclude=/dists/**/installer-armhf/ --exclude=/dists/**/Contents-armhf.gz --exclude=/dists/**/Contents-udeb-armhf.gz --exclude=/dists/**/Contents-armhf.diff/ --exclude=/indices/**/arch-armhf.files --exclude=/indices/**/arch-armhf.list.gz --exclude=/pool/**/*_armhf.deb --exclude=/pool/**/*_armhf.udeb --exclude=/pool/**/*_armhf.changes --exclude=/dists/**/binary-hppa/ --exclude=/dists/**/installer-hppa/ --exclude=/dists/**/Contents-hppa.gz --exclude=/dists/**/Contents-udeb-hppa.gz --exclude=/dists/**/Contents-hppa.diff/ --exclude=/indices/**/arch-hppa.files --exclude=/indices/**/arch-hppa.list.gz --exclude=/pool/**/*_hppa.deb --exclude=/pool/**/*_hppa.udeb --exclude=/pool/**/*_hppa.changes --exclude=/dists/**/binary-hurd-i386/ --exclude=/dists/**/installer-hurd-i386/ --exclude=/dists/**/Contents-hurd-i386.gz --exclude=/dists/**/Contents-udeb-hurd-i386.gz --exclude=/dists/**/Contents-hurd-i386.diff/ --exclude=/indices/**/arch-hurd-i386.files --exclude=/indices/**/arch-hurd-i386.list.gz --exclude=/pool/**/*_hurd-i386.deb --exclude=/pool/**/*_hurd-i386.udeb --exclude=/pool/**/*_hurd-i386.changes --exclude=/dists/**/binary-ia64/ --exclude=/dists/**/installer-ia64/ --exclude=/dists/**/Contents-ia64.gz --exclude=/dists/**/Contents-udeb-ia64.gz --exclude=/dists/**/Contents-ia64.diff/ --exclude=/indices/**/arch-ia64.files --exclude=/indices/**/arch-ia64.list.gz --exclude=/pool/**/*_ia64.deb --exclude=/pool/**/*_ia64.udeb --exclude=/pool/**/*_ia64.changes --exclude=/dists/**/binary-kfreebsd-amd64/ --exclude=/dists/**/installer-kfreebsd-amd64/ --exclude=/dists/**/Contents-kfreebsd-amd64.gz --exclude=/dists/**/Contents-udeb-kfreebsd-amd64.gz --exclude=/dists/**/Contents-kfreebsd-amd64.diff/ --exclude=/indices/**/arch-kfreebsd-amd64.files --exclude=/indices/**/arch-kfreebsd-amd64.list.gz --exclude=/pool/**/*_kfreebsd-amd64.deb --exclude=/pool/**/*_kfreebsd-amd64.udeb --exclude=/pool/**/*_kfreebsd-amd64.changes --exclude=/dists/**/binary-kfreebsd-i386/ --exclude=/dists/**/installer-kfreebsd-i386/ --exclude=/dists/**/Contents-kfreebsd-i386.gz --exclude=/dists/**/Contents-udeb-kfreebsd-i386.gz --exclude=/dists/**/Contents-kfreebsd-i386.diff/ --exclude=/indices/**/arch-kfreebsd-i386.files --exclude=/indices/**/arch-kfreebsd-i386.list.gz --exclude=/pool/**/*_kfreebsd-i386.deb --exclude=/pool/**/*_kfreebsd-i386.udeb --exclude=/pool/**/*_kfreebsd-i386.changes --exclude=/dists/**/binary-m68k/ --exclude=/dists/**/installer-m68k/ --exclude=/dists/**/Contents-m68k.gz --exclude=/dists/**/Contents-udeb-m68k.gz --exclude=/dists/**/Contents-m68k.diff/ --exclude=/indices/**/arch-m68k.files --exclude=/indices/**/arch-m68k.list.gz --exclude=/pool/**/*_m68k.deb --exclude=/pool/**/*_m68k.udeb --exclude=/pool/**/*_m68k.changes --exclude=/dists/**/binary-mipsel/ --exclude=/dists/**/installer-mipsel/ --exclude=/dists/**/Contents-mipsel.gz --exclude=/dists/**/Contents-udeb-mipsel.gz --exclude=/dists/**/Contents-mipsel.diff/ --exclude=/indices/**/arch-mipsel.files --exclude=/indices/**/arch-mipsel.list.gz --exclude=/pool/**/*_mipsel.deb --exclude=/pool/**/*_mipsel.udeb --exclude=/pool/**/*_mipsel.changes --exclude=/dists/**/binary-mips/ --exclude=/dists/**/installer-mips/ --exclude=/dists/**/Contents-mips.gz --exclude=/dists/**/Contents-udeb-mips.gz --exclude=/dists/**/Contents-mips.diff/ --exclude=/indices/**/arch-mips.files --exclude=/indices/**/arch-mips.list.gz --exclude=/pool/**/*_mips.deb --exclude=/pool/**/*_mips.udeb --exclude=/pool/**/*_mips.changes --exclude=/dists/**/binary-s390/ --exclude=/dists/**/installer-s390/ --exclude=/dists/**/Contents-s390.gz --exclude=/dists/**/Contents-udeb-s390.gz --exclude=/dists/**/Contents-s390.diff/ --exclude=/indices/**/arch-s390.files --exclude=/indices/**/arch-s390.list.gz --exclude=/pool/**/*_s390.deb --exclude=/pool/**/*_s390.udeb --exclude=/pool/**/*_s390.changes --exclude=/dists/**/binary-s390x/ --exclude=/dists/**/installer-s390x/ --exclude=/dists/**/Contents-s390x.gz --exclude=/dists/**/Contents-udeb-s390x.gz --exclude=/dists/**/Contents-s390x.diff/ --exclude=/indices/**/arch-s390x.files --exclude=/indices/**/arch-s390x.list.gz --exclude=/pool/**/*_s390x.deb --exclude=/pool/**/*_s390x.udeb --exclude=/pool/**/*_s390x.changes --exclude=/dists/**/binary-sh/ --exclude=/dists/**/installer-sh/ --exclude=/dists/**/Contents-sh.gz --exclude=/dists/**/Contents-udeb-sh.gz --exclude=/dists/**/Contents-sh.diff/ --exclude=/indices/**/arch-sh.files --exclude=/indices/**/arch-sh.list.gz --exclude=/pool/**/*_sh.deb --exclude=/pool/**/*_sh.udeb --exclude=/pool/**/*_sh.changes --exclude=/dists/**/binary-sparc/ --exclude=/dists/**/installer-sparc/ --exclude=/dists/**/Contents-sparc.gz --exclude=/dists/**/Contents-udeb-sparc.gz --exclude=/dists/**/Contents-sparc.diff/ --exclude=/indices/**/arch-sparc.files --exclude=/indices/**/arch-sparc.list.gz --exclude=/pool/**/*_sparc.deb --exclude=/pool/**/*_sparc.udeb --exclude=/pool/**/*_sparc.changes mirror.linux.org.au::debian /mirrors/debian/
Oct 14 14:11:23 debian ftpsync[19935]: Back from rsync with returncode 0
Oct 14 14:11:23 debian ftpsync[19935]: Latest recorded rsync transfer speed: 418 KB/s
Oct 14 14:11:23 debian ftpsync[19935]: Running stage2: rsync --filter=exclude_/Archive-Update-in-Progress-debian.localdomain --filter=protect_/Archive-Update-in-Progress-debian.localdomain --filter=exclude_/Archive-Update-Required-debian.localdomain --filter=protect_/Archive-Update-Required-debian.localdomain --filter=exclude_/project/trace/debian.localdomain --filter=protect_/project/trace/debian.localdomain --filter=exclude_/project/trace/debian.localdomain-stage1 --filter=protect_/project/trace/debian.localdomain-stage1 --filter=exclude_/project/trace/_hierarchy --filter=protect_/project/trace/_hierarchy --filter=exclude_/project/trace/_traces --filter=protect_/project/trace/_traces --filter=include_/project/ --filter=protect_/project/ --filter=include_/project/trace/ --filter=protect_/project/trace/ --filter=include_/project/trace/* --bwlimit=0 -prltvHSB8192 --timeout 3600 --stats --max-delete=40000 --delay-updates --delete --delete-excluded --delete-delay --exclude=.~tmp~/ --exclude=/dists/**/binary-alpha/ --exclude=/dists/**/installer-alpha/ --exclude=/dists/**/Contents-alpha.gz --exclude=/dists/**/Contents-udeb-alpha.gz --exclude=/dists/**/Contents-alpha.diff/ --exclude=/indices/**/arch-alpha.files --exclude=/indices/**/arch-alpha.list.gz --exclude=/pool/**/*_alpha.deb --exclude=/pool/**/*_alpha.udeb --exclude=/pool/**/*_alpha.changes --exclude=/dists/**/binary-arm/ --exclude=/dists/**/installer-arm/ --exclude=/dists/**/Contents-arm.gz --exclude=/dists/**/Contents-udeb-arm.gz --exclude=/dists/**/Contents-arm.diff/ --exclude=/indices/**/arch-arm.files --exclude=/indices/**/arch-arm.list.gz --exclude=/pool/**/*_arm.deb --exclude=/pool/**/*_arm.udeb --exclude=/pool/**/*_arm.changes --exclude=/dists/**/binary-armhf/ --exclude=/dists/**/installer-armhf/ --exclude=/dists/**/Contents-armhf.gz --exclude=/dists/**/Contents-udeb-armhf.gz --exclude=/dists/**/Contents-armhf.diff/ --exclude=/indices/**/arch-armhf.files --exclude=/indices/**/arch-armhf.list.gz --exclude=/pool/**/*_armhf.deb --exclude=/pool/**/*_armhf.udeb --exclude=/pool/**/*_armhf.changes --exclude=/dists/**/binary-hppa/ --exclude=/dists/**/installer-hppa/ --exclude=/dists/**/Contents-hppa.gz --exclude=/dists/**/Contents-udeb-hppa.gz --exclude=/dists/**/Contents-hppa.diff/ --exclude=/indices/**/arch-hppa.files --exclude=/indices/**/arch-hppa.list.gz --exclude=/pool/**/*_hppa.deb --exclude=/pool/**/*_hppa.udeb --exclude=/pool/**/*_hppa.changes --exclude=/dists/**/binary-hurd-i386/ --exclude=/dists/**/installer-hurd-i386/ --exclude=/dists/**/Contents-hurd-i386.gz --exclude=/dists/**/Contents-udeb-hurd-i386.gz --exclude=/dists/**/Contents-hurd-i386.diff/ --exclude=/indices/**/arch-hurd-i386.files --exclude=/indices/**/arch-hurd-i386.list.gz --exclude=/pool/**/*_hurd-i386.deb --exclude=/pool/**/*_hurd-i386.udeb --exclude=/pool/**/*_hurd-i386.changes --exclude=/dists/**/binary-ia64/ --exclude=/dists/**/installer-ia64/ --exclude=/dists/**/Contents-ia64.gz --exclude=/dists/**/Contents-udeb-ia64.gz --exclude=/dists/**/Contents-ia64.diff/ --exclude=/indices/**/arch-ia64.files --exclude=/indices/**/arch-ia64.list.gz --exclude=/pool/**/*_ia64.deb --exclude=/pool/**/*_ia64.udeb --exclude=/pool/**/*_ia64.changes --exclude=/dists/**/binary-kfreebsd-amd64/ --exclude=/dists/**/installer-kfreebsd-amd64/ --exclude=/dists/**/Contents-kfreebsd-amd64.gz --exclude=/dists/**/Contents-udeb-kfreebsd-amd64.gz --exclude=/dists/**/Contents-kfreebsd-amd64.diff/ --exclude=/indices/**/arch-kfreebsd-amd64.files --exclude=/indices/**/arch-kfreebsd-amd64.list.gz --exclude=/pool/**/*_kfreebsd-amd64.deb --exclude=/pool/**/*_kfreebsd-amd64.udeb --exclude=/pool/**/*_kfreebsd-amd64.changes --exclude=/dists/**/binary-kfreebsd-i386/ --exclude=/dists/**/installer-kfreebsd-i386/ --exclude=/dists/**/Contents-kfreebsd-i386.gz --exclude=/dists/**/Contents-udeb-kfreebsd-i386.gz --exclude=/dists/**/Contents-kfreebsd-i386.diff/ --exclude=/indices/**/arch-kfreebsd-i386.files --exclude=/indices/**/arch-kfreebsd-i386.list.gz --exclude=/pool/**/*_kfreebsd-i386.deb --exclude=/pool/**/*_kfreebsd-i386.udeb --exclude=/pool/**/*_kfreebsd-i386.changes --exclude=/dists/**/binary-m68k/ --exclude=/dists/**/installer-m68k/ --exclude=/dists/**/Contents-m68k.gz --exclude=/dists/**/Contents-udeb-m68k.gz --exclude=/dists/**/Contents-m68k.diff/ --exclude=/indices/**/arch-m68k.files --exclude=/indices/**/arch-m68k.list.gz --exclude=/pool/**/*_m68k.deb --exclude=/pool/**/*_m68k.udeb --exclude=/pool/**/*_m68k.changes --exclude=/dists/**/binary-mipsel/ --exclude=/dists/**/installer-mipsel/ --exclude=/dists/**/Contents-mipsel.gz --exclude=/dists/**/Contents-udeb-mipsel.gz --exclude=/dists/**/Contents-mipsel.diff/ --exclude=/indices/**/arch-mipsel.files --exclude=/indices/**/arch-mipsel.list.gz --exclude=/pool/**/*_mipsel.deb --exclude=/pool/**/*_mipsel.udeb --exclude=/pool/**/*_mipsel.changes --exclude=/dists/**/binary-mips/ --exclude=/dists/**/installer-mips/ --exclude=/dists/**/Contents-mips.gz --exclude=/dists/**/Contents-udeb-mips.gz --exclude=/dists/**/Contents-mips.diff/ --exclude=/indices/**/arch-mips.files --exclude=/indices/**/arch-mips.list.gz --exclude=/pool/**/*_mips.deb --exclude=/pool/**/*_mips.udeb --exclude=/pool/**/*_mips.changes --exclude=/dists/**/binary-s390/ --exclude=/dists/**/installer-s390/ --exclude=/dists/**/Contents-s390.gz --exclude=/dists/**/Contents-udeb-s390.gz --exclude=/dists/**/Contents-s390.diff/ --exclude=/indices/**/arch-s390.files --exclude=/indices/**/arch-s390.list.gz --exclude=/pool/**/*_s390.deb --exclude=/pool/**/*_s390.udeb --exclude=/pool/**/*_s390.changes --exclude=/dists/**/binary-s390x/ --exclude=/dists/**/installer-s390x/ --exclude=/dists/**/Contents-s390x.gz --exclude=/dists/**/Contents-udeb-s390x.gz --exclude=/dists/**/Contents-s390x.diff/ --exclude=/indices/**/arch-s390x.files --exclude=/indices/**/arch-s390x.list.gz --exclude=/pool/**/*_s390x.deb --exclude=/pool/**/*_s390x.udeb --exclude=/pool/**/*_s390x.changes --exclude=/dists/**/binary-sh/ --exclude=/dists/**/installer-sh/ --exclude=/dists/**/Contents-sh.gz --exclude=/dists/**/Contents-udeb-sh.gz --exclude=/dists/**/Contents-sh.diff/ --exclude=/indices/**/arch-sh.files --exclude=/indices/**/arch-sh.list.gz --exclude=/pool/**/*_sh.deb --exclude=/pool/**/*_sh.udeb --exclude=/pool/**/*_sh.changes --exclude=/dists/**/binary-sparc/ --exclude=/dists/**/installer-sparc/ --exclude=/dists/**/Contents-sparc.gz --exclude=/dists/**/Contents-udeb-sparc.gz --exclude=/dists/**/Contents-sparc.diff/ --exclude=/indices/**/arch-sparc.files --exclude=/indices/**/arch-sparc.list.gz --exclude=/pool/**/*_sparc.deb --exclude=/pool/**/*_sparc.udeb --exclude=/pool/**/*_sparc.changes mirror.linux.org.au::debian /mirrors/debian/
Oct 14 14:15:52 debian ftpsync[19935]: Back from rsync with returncode 0
Oct 14 14:15:52 debian ftpsync[19935]: Latest recorded rsync transfer speed: 161 KB/s
Oct 14 14:16:13 debian ftpsync[19935]: Mirrorsync done



Thanks
Steve



-----Original Message-----
From: Bastian Blank [mailto:waldi@debian.org] 
Sent: Friday, 13 October 2017 4:29 AM
To: debian-mirrors@lists.debian.org
Subject: Re: rsync errors when using latest ftpsync

Hi Steve

On Thu, Oct 12, 2017 at 07:43:39AM +1100, Steve (IWWD) wrote:
> I'm receiving the following errors while running the latest version of
> ftpsync on our mirror:

> rsync: link
> "/mirrors/debian/dists/sid/main/installer-amd64/20170828/images/cdrom/xen/in
> itrd.gz" =>
> dists/buster/main/installer-amd64/20170828/images/cdrom/xen/initrd.gz
> failed: Operation not permitted (1)

This looks odd.  This file is actually a symlink, but got two names.

> There doesn't seem to be any permissions issues, but the file it's
> attempting to link to is also a link to another location.

Permission problems would be EACCESS or "Permission denied".  "Operation
not permitted" is something that can't be done for other reasons.

> I'm unable to find any resources which indicate a similar issue or
> resolution.  Is anyone able to point me in the right direction?

What kind of filesystem do you use?

Are all parts of the mirror tree owned by the same user?  There may be
some hardlink protection in place that forbids hard-links for files the
user can't write.  As this is a symlink, rsync may not be able to change
the owner.

Regards,
Bastian

-- 
It is a human characteristic to love little animals, especially if
they're attractive in some way.
		-- McCoy, "The Trouble with Tribbles", stardate 4525.6