Bug#685407: [wheezy] ext4 dir_index + nfs duplicate cookies problem with large dovecot maildirs

[Brian Kroth <bpkroth@gmail.com>]

Jonathan Nieder <jrnieder@gmail.com> 2012-08-20 09:39:
> Hi,
>
> Brian Paul Kroth wrote:
>
> > - multiple ext4 filesystems with dir_index enabled for maildir account storage
> > - serves these over nfs (v3 or v4, doesn't matter) to server clients
> > running dovecot to serve imap to real clients
> > - "ls -l | wc -l" or readdir() in directories with many entries
> > (I've   seen it with as few as ~7000) results in the "duplicate
> > cookie" or   "too many levels of symbolic links" error being
> > returned and the   directory listing will be cut short.
>
> Some patches intended to address this on the server side were
> merged during the 3.4 merge window[*].  Could you try 3.5.2 or
> newer from experimental and let us know how it goes?
>
> Curious,
> Jonathan
>
> [*] http://thread.gmane.org/gmane.linux.file-systems/60157

linux-image-3.5-trunk-amd64 (3.5-1~experimental.1) on the server does 
indeed appear to fix this.  NFSv3 and NFSv4 tested from the following 
client kernels:
- linux-image-3.2.0-0.bpo.2-amd64 (3.2.20-1~bpo60+1)
- linux-image-2.6.39-bpo.2-amd64 (2.6.39-3~bpo60+1)
- linux-image-2.6.32-5-amd64 (2.6.32-45)

Attached is a simple script I used to create a test dir for this.  The 
ext4 volume was 3G.

Any chance this will be backported to squeeze as well?

Thanks,
Brian

#!/usr/bin/perl -w

use strict;

my $top = 305388;	# starts to cause symbolic link loop warnings here
my $dir = './test';

if (! -d $dir) {
	mkdir($dir) or die($!);
}

for my $i (1..$top) {
	my $file = "$dir/test-$i";
	
	if ($i % 1000 == 0) {	# show some progress
		print $file, "\n";
	}

	next if (-f $file);

	open(FILE, '>', $file) or die($!);
	print FILE "test-$i";
	close(FILE) or die($!);
}

# Now NFS export it, mount it elsewhere, and "ls -l | wc -l" or readdir() from
# that elsewhere to see if it complains.

Attachment: signature.asc
Description: Digital signature