W dniu 22.08.2022 o 10:20, Florian Weimer pisze:
- Marcin Juszkiewicz:
As part of OpenStack deployments we deploy RabbitMQ. During current cycle I looked at moving from CentOS Stream 8 to 9.
And RabbitMQ is a problem.
When I boot CentOS Stream 9 system and then use just built 'rabbitmq' container memory use of "/usr/lib64/erlang/erts-12.3.2.2/bin/beam.smp" process goes up to 1.6GB ram:
If I boot the same container on Debian host then same process uses 0.2GB ram:
I looked at this briefly and it seems that th Erlang runtime does its own memory accounting, so it's probably not related to different kernel MM defaults (THP settings in particular). But the Erlang code is a bit impenetrable to me, so I could be wrong about it.
It could still be related to auto-tuning based on Erlang's interpretation of system/cgroup parameters.
Have you confirmed that the numbers are real? That for a given amount of RAM in a VM (say 8 GB), you can 30 containers at the same time on Debian, and only 4 on CentOS?
The Erlang JIT likely uses the multi-mapping kludge in SELinux mode, and that code could have a leak or a memory accounting bug.
But I think someone who is familiar with the Erlang runtime needs to look at this.
Turned out that this is due to amount of available file descriptors...
https://github.com/rabbitmq/erlang-rpm/discussions/104#discussioncomment-345...
EL9 (both CentOS Stream 9 and Rocky Linux 9) have 1073741816 while CentOS Stream 8 has 1048576 - after lowering it's acceptable memory usage.