On 2017-12-01 18:56, Pascal wrote: > hello, > > while doing some tests on the ntfs file system, I met some strange things > with my qcow2 disk images. > the images are on a partition mounted in tmpfs, but the result is the same > when they are recorded on a partition in ext4 format. > I don't know where the problem comes from : qemu-nbd or format qcow2 or > something else ? > do not hesitate if you want more informations. I see the issue here as well (and with raw, too). tl;dr: Seems like a kernel issue to me (CC-ing the NBD list because that's the best I can do). When tracing the accesses, it appears that at least the NTFS header is not read from the source disk when copying the data over. I would guess this is due to caching, because Linux has read that sector before the mkfs.ntfs (so it was zero then). And the issue disappears if I insert a "blockdev --flushbufs /dev/nbd0" after the mkfs.ntfs -- but not if I flush nbd0p1, interestingly. I would guess the kernel has different caches for the whole device and each partition? Well, that's nice. Not sure if that is a bug or whether that is just how it is... (But nbd just uses the normal blockdev partitioning, so I guess it's by design? (And the same issue appears with kpartx, too)) You can also see this on the source volume alone: $ qemu-img create [...] # qemu-nbd -c /dev/nbd0 [...] # fdisk /dev/nbd0 # mkfs.ntfs /dev/nbd0p1 # hexdump -C /dev/nbd0 [...] 00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00102000 ff ff 00 07 00 00 00 00 00 00 00 00 00 00 00 00 |................|[...] # blockdev --flushbufs /dev/nbd0 # hexdump -C /dev/nbd0 [...] 00100000 eb 52 90 4e 54 46 53 20 20 20 20 00 02 08 00 00 |.R.NTFS .....| 00100010 00 00 00 00 00 f8 00 00 00 00 00 00 00 00 00 00 |................| [...] However, there is still an open question: I can't reproduce this with loop or real devices. I only see this with NBD. Why? After having dug for too long into the kernel sources, my best guess right now is that the kernel NBD driver might be missing some necessary flushes. Whenever one NBD device is accessed (through a partition or not), it is necessary to flush all device nodes that are associated with it -- but loop doesn't seem to be doing this, and I would expect the general partition framework to handle this already. Therefore, my best guess is a bad guess. But note that I can reproduce the issue with nbd-server and nbd-client just fine: # dd if=/dev/zero of=/tmp/foo.img bs=1M count=2048 2048+0 records in 2048+0 records out 2147483648 bytes (2.1 GB, 2.0 GiB) copied, 0.517711 s, 4.1 GB/s # nbd-server 10809 /tmp/foo.img ** (process:17331): WARNING **: Specifying an export on the command line no longer uses the oldstyle protocol. # nbd-client localhost /dev/nbd0 Warning: the oldstyle protocol is no longer supported. This method now uses the newstyle protocol with a default export Negotiation: ..size = 2048MB bs=1024, sz=2147483648 bytes # echo -e 'n\n\n\n\n\nt\n7\nw' | fdisk /dev/nbd0 [...] # mkfs.ntfs /dev/nbd0p1 [...] # hexdump -C /dev/nbd0 | less [...] 00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00102000 ff ff 00 07 00 00 00 00 00 00 00 00 00 00 00 00 |................|[...] # blockdev --flushbufs /dev/nbd0 # hexdump -C /dev/nbd0 | less [...] 00100000 eb 52 90 4e 54 46 53 20 20 20 20 00 04 04 00 00 |.R.NTFS .....| 00100010 00 00 00 00 00 f8 00 00 00 00 00 00 00 00 00 00 |................|[...] Max
Attachment:
signature.asc
Description: OpenPGP digital signature