Re: Too many open files

[Greg Wooledge <greg@wooledge.org>]

On Mon, Aug 18, 2025 at 21:52:30 -0700, Ken Mankoff wrote:
> ulimit -Hn and -Sn both report 32768, so that's not it.

Those are per-process limits, not global.  More on this in a bit.

> $ cat /proc/sys/fs/file-max
> 1000000
> 
> Also seen at
> 
> $ cat /proc/sys/fs/file-nr
> 25312   0       1000000
> 
> Seems reasonable so far, and nothing obvious (to me) in dmesg.

That number actually looks suspiciously low *and* artificial, though I
agree that it appears you're not bumping into that limit at the moment.

hobbit:~$ cat /proc/sys/fs/file-max
9223372036854775807 

hobbit:~$ cat /proc/sys/fs/file-nr
11560 0 9223372036854775807 

Is yours being set by a configuration file somewhere?  Mine is whatever
the system chose; I never touched it.  Even on a completely different
system (OpenVZ container) with a much older kernel, the system-wide
open file limit is much higher than yours:

root@remote:~# cat /proc/sys/fs/file-nr
2016    0       46542663 

However, what we really need to know is whether you're hitting the
system-wide limit, or the per-process limit.  Looking at open(2), there
are two separate errors:

       EMFILE The per-process limit on the number of open file descriptors  has
              been  reached  (see  the  description  of RLIMIT_NOFILE in
              getrlimit(2)).

       ENFILE The system-wide limit on the total number of open files has  been
              reached.

And their canonical error messages are:

/usr/include/asm-generic/errno-base.h:#define	ENFILE		23	/* File table overflow */
/usr/include/asm-generic/errno-base.h:#define	EMFILE		24	/* Too many open files */

If you're seeing the message "Too many open files", you're *probably*
hitting EMFILE (the per-process limit) rather than ENFILE (system-wide),
but it would be helpful to see that definitively, e.g. with strace.

If you are hitting EMFILE, as I suspect, then all of your investigation
with (for example) sudo | awk | sort is not helpful, because that's
looking at the system-wide state, when you need to be looking at a
single process at a time.

On my newly-upgraded-to-trixie system, my per-process hard limit is
quite a bit higher than yours:

hobbit:~$ ulimit -Hn
524288
hobbit:~$ ulimit -Sn
1024 

I'm not sure how significant that is.  Even your 32k seems very reasonable
for most processes.  Of course, you only looked at the limits for *one*
process, and that was an interactive shell.  Other processes that are
spawned in other ways may have very different resource limits.