Stack size

Applications using POSIX threads (a.k.a. NPTL threads, as implemented by glibc) reserves some amount of virtual memory for the stack per each thread. Unfortunately, glibc uses the maximum stack size (as configured by ulimit -s a.k.a. setrlimit(RLIMIT_STACK, ...) as the amount of memory to be reserved. In most of the new distributions this parameter is set to a very high value of 10240 (1K blocks), i.e. 10 megabytes. Therefore, glibc reserves 10MB of anonymous virtual memory per each thread, which can easily lead to privvmpages shortage if it's not set high enough.

Getting the current stack size limit

You can see the current stack size limit (RLIMIT_STACK) by running ulimit -s command. Result is in kilobytes.

Increasing privvmpages

The obvious solution is to increase privvmpages value by the amount enough for the given number of threads and the current value of RLIMIT_STACK. For example, if you plan to run 10 threads and stack size limit is set to 10240, you need to increase privvmpages for 10240*10/4, i.e. 25600 (4 is a page size, in kilobytes).

Decreasing the maximum stack size

Most applications do not need a stack as big as 10 megabytes, so you can set it to a smaller value before starting your threaded app. One obvious place for doing that is application startup script, like /etc/init.d/mysqld or /etc/init.d/httpd. What you need to do is to add a line like

ulimit -s 1024 # Stack size limit is 1M

to the beginning of such a script.