Hi Daniel!
Apologies for the late reply. I had seen your message back then but it
somehow fell of the table and eventually forgot to answer. I just now
found your report again since I resumed working on the hfsprogs
package.
On Mon, 2021-09-27 at 12:45 +0200, Daniel Höpfl wrote:
Apple (the upstream) never finished the implementation of the
directory hardlink support
in fsck_hfs. Since, once again, my TimeMachine backup rendered
unrepairable due to directory
hardlink errors, I implemented what was required to fix the errors I
encountered.
The current hfsprogs package in Debian is based on a patched version
540.1 of Apple's
»diskdev_cmds« package [1] while Apple has actually continued their
work on the HFS code.
in that package. However, after version 557.3.1 Apple decided to move
the HFS code into a
separate package called just »hfs« [2] which makes sense given the fact
that Apple switched
their default filesystem to APFS instead and diskdev_cms is no longer
strictly tied to HFS.
I have started working on an updated Debian hfsprogs package which is
based on the latest version
556.100.11 of the hfs package [3]. The code already compiles in
openSUSE [4] but I don't consider
the Linux port to be complete yet.
However, you may want to try whether the latest version of Apple's own
fsck_hfs has support for
fixing hardlink problems. If not, you may want to rebase your patch on
top of hfs 560.100.11.
My trust in Apple's feedback system is very low so I am sending the
patch to you in the hopes
that you can forward it to Apple.
Try opening a pull request on Github [5]. Apple has actually moved
their open source projects to
Github and it seems they're actually accepting pull requests for some
projects.
The patch I added should work, with offsets, on both, Debian's sources
as well as Apple's sources.
My changes:
- Any hardlink inode with a parent other than the metadata directory
is changed
to have the correct parent.
I have seen these. I can understand how this happens when a normal
directory
is hardlined for the first time. The journal SHOULD prevent this
from being a
problem but obviously this is not the case.
- Directory hardlink inodes named "temp..." get deleted.
These directories were deleted while open when the volume was
removed without
unmounting. Since these directories have already been deleted,
deleting them
in fsck is the right thing to do. (Documented like that for files in
the HFS+
spec.)
- Directory hardlink inodes with names other than "temp..." or
"dir_..." are moved
to "lost+found".
It would be better to rename them to "dir_..." in a first pass,
there might still
be valid links to them. These links are deleted by a different
repair step and I
did not see how to remove those other repair steps. Would probably
lead to a bigger
rewrite of the hardlink check code so I decided to just move these
to l+f.
I have never seen this anyways.
After all, none of the errors should happen in a journaled filesystem
(directory
hardlinks require the Journal to be active). My guess is that using
HFS+ over AFP
(as TimeMachine does) does not correctly persist the journal before
persisting changes.
Thus loosing the connection during directory hardlink operation can
result in partially
executed atomic operation.
Sounds like a very useful and welcome improvement. I will definitely
incorporate this
patch once I'm ready with the new upstream version. But maybe we're
lucky and you can
get your changes merged upstream with Apple in the mean time.
FWIW, I have also reached out to Apple Legal trying to convince them to
relicense the
hfs package to a more permissive license such as the Apache License
like they have done
for other projects such as mDNSResponder [6]. Apple's own APSL license
is considered
non-free by Debian [7] which makes redistribution of the package more
complicated.
I will attach the patch and test cases for each of the three fixed
errors. These test
cases have been constructed in a hex editor, they might not be what a
real error looks
like. The first two repairs successfully repaired my broken backup
file so I guess
they work (sometimes).
Thanks, having test cases will be very useful for verifying my patched
hfs package!
Adrian
[1] https://github.com/apple-opensource/diskdev_cmds/tree/540.1
[2] https://github.com/apple-opensource/hfs
[3] https://github.com/glaubitz/hfs/tree/linux
[4]
https://build.opensuse.org/package/show/home:glaubitz:branches:filesystems/hfsplus-tools
[5] https://github.com/apple-opensource/hfs
[6] https://github.com/apple-oss-distributions/mDNSResponder/
[7]
https://wiki.debian.org/DFSGLicenses#Apple_Public_Source_License_.28APSL.29