Error: EMFILE: too many open files, watch, unless I use sudo

Question

Description

Recently I've run into an problem. I am not able to run yarn start in element-web directory, I get these errors. Originally I thought it had something to do with element-web itself so I created an issue. Some time after that I tried to run wintersmith preview in bibviz directory and got the same errors. This was weird so I tried to create an Angular project and run ng serve and errors again. I headed to the issue to close it as it wasn't an element-web issue. I found that there was another issue created with the same problem. It had already been closed by turt2live saying it looks like you've run out of memory on your system. Based on this I tried to turn of most programs running in the background and now all the commands worked.

I am sure that ng serve used to work in the past.

My PC has 16 GB of RAM and the commands already fail when I am on 7/16 GB. I can't see any memory spikes when running the commands. Running the commands with sudo also completely eliminates the problem. This doesn't make any sense to me.

Research lead me to ulimits but they seem to have no effect. I have also installed watchman with no effect.

Can someone tell me what I am missing?

Thank you in advance!

Info

I am on Debian 11 Bullseye. This is the output of a few commands that could be useful.

As a regular user:

> uname -a
Linux Simon-s-PC 5.8.0-3-amd64 #1 SMP Debian 5.8.14-1 (2020-10-10) x86_64 GNU/Linux
> sudo sysctl fs.inotify.max_user_watches
fs.inotify.max_user_watches = 524288
> ulimit -a
-t: cpu time (seconds)              unlimited
-f: file size (blocks)              unlimited
-d: data seg size (kbytes)          unlimited
-s: stack size (kbytes)             8192
-c: core file size (blocks)         0
-m: resident set size (kbytes)      unlimited
-u: processes                       46482
-n: file descriptors                8192
-l: locked-in-memory size (kbytes)  unlimited
-v: address space (kbytes)          unlimited
-x: file locks                      unlimited
-i: pending signals                 63664
-q: bytes in POSIX msg queues       819200
-e: max nice                        0
-r: max rt priority                 95
-N 15:                              unlimited
> yarn --version
1.22.5

With sudo su:

> sysctl fs.inotify.max_user_watches
fs.inotify.max_user_watches = 524288
> ulimit -a
-t: cpu time (seconds)              unlimited
-f: file size (blocks)              unlimited
-d: data seg size (kbytes)          unlimited
-s: stack size (kbytes)             8192
-c: core file size (blocks)         0
-m: resident set size (kbytes)      unlimited
-u: processes                       63664
-n: file descriptors                1024
-l: locked-in-memory size (kbytes)  2043392
-v: address space (kbytes)          unlimited
-x: file locks                      unlimited
-i: pending signals                 63664
-q: bytes in POSIX msg queues       819200
-e: max nice                        0
-r: max rt priority                 0
-N 15:                              unlimited

Answer 1

I think I've found a solution:

• Set limits in /etc/sysctl.conf by adding:

fs.inotify.max_user_watches=524288
fs.inotify.max_user_instances=512

• Open a new terminal or reload sysctl.conf variables with

  sudo sysctl --system
  

• Run yarn start

Everything should work fine now, hopefully. If it doesn't work try setting the limits higher.