Dropped Connections and "Failed to create cgroup nnnn: -17" Kernel Message When vsftpd Spawning a New Process
Hello list,
I am experiencing the following issue with a Debian squeeze based server
and the most recent squeeze-backports kernel:
I realized that some vstfpd daemons randomly drop connections (sending a
FIN right after the initial TCP hand shake was completed). Furthermore,
a "Failed to create cgroup nnnn: -17" message is logged by the kernel.
Furthermore, I am observing a steadily increasing number of directories
named like pids being created in the root of the cgroup virtual
filesystem (mounted at /cgroup). For each connection attempt to a vsftpd
daemon a new directory is created. Those directories seem to be never
deleted. After a few days of uptime there are about 7,500 directories
while there constantly are only about 150 processes running (more or
less idling, this server usually has low load).
When stracing vsftpd the call that fails seems to be this one (full
output below):
clone(child_stack=0, flags=0x28000000|SIGCHLD) = -1 EEXIST (File exists)
Which makes me believe that those "zombie directories" in /cgroup might
conflict with the new pid . The longer the server is up the more likely
it becomes that connections are dropped.
Side note: The affected vsftpd daemons are running on a server that also
is hosting an LXC-based virtual server. I have experienced a steadily
increasing soft IRQ load on the server while a cgroup virtual filesystem
being mounted. I have upgraded to the recent squeeze-backports kernel
which seems not to suffer from this soft IRQ issue. vsftpd daemons
running inside LXC containers do not drop connections.
Anyone experiencing a similar issue or has any suggestions? When
reporting a bug for this issue would this need to be reported against
the kernel package?
Below is some information I thought might be useful. If required, I will
gladly provide any additional information.
Cheers,
Dirk
I am using vsftpd 2.3.2-3 which AFAIK is the most recent version
available from the squeeze and squeeze-backports repositories.
"uname -a" output:
Linux xxxxx 2.6.39-bpo.2-amd64 #1 SMP Tue Jul 26 10:35:23 UTC 2011
x86_64 GNU/Linux
"strace vsftpd /etc/vsftp.conf" output (successful connection attempt):
alarm(1) = 0
rt_sigreturn(0x1) = -1 EINTR (Interrupted system call)
alarm(0) = 1
wait4(-1, NULL, WNOHANG, NULL) = 6385
wait4(-1, NULL, WNOHANG, NULL) = -1 ECHILD (No child processes)
accept(3, {sa_family=AF_INET, sin_port=htons(46631),
sin_addr=inet_addr("xxx.xxx.xxx.xxx")}, [16]) = 4
clone(child_stack=0, flags=0x28000000|SIGCHLD) = 6387
close(4) = 0
accept(3, 0x7fffc3ecdf70, [28]) = ? ERESTARTSYS (To be restarted)
"strace vsftpd /etc/vsftp.conf" output (failed connection attempt):
alarm(1) = 0
rt_sigreturn(0x1) = -1 EINTR (Interrupted system call)
alarm(0) = 1
wait4(-1, NULL, WNOHANG, NULL) = 6387
wait4(-1, NULL, WNOHANG, NULL) = -1 ECHILD (No child processes)
accept(3, {sa_family=AF_INET, sin_port=htons(47917),
sin_addr=inet_addr("xxx.xxx.xxx.xxx")}, [16]) = 4
clone(child_stack=0, flags=0x28000000|SIGCHLD) = -1 EEXIST (File exists)
close(4) = 0
accept(33, 0x7fffc3ecdf70, [28]) = ? ERESTARTSYS (To be restarted)
Reply to: