Apache auth_digest:error No space left on device

My Apache machine just took a dump :(  And on the restart it presented these errors:

[auth_digest:notice] [pid 29978] AH01757: generating secret for digest authentication ...
[core:emerg] [pid 29978] (28)No space left on device: AH00023: Couldn't create the authdigest-opaque mutex 
[auth_digest:error] [pid 29978] (28)No space left on device: AH01760: failed to create lock (opaque_lock) - all nonce-count checking, one-time nonces, and MD5-sess algorithm disabled
[:emerg] [pid 29978] AH00020: Configuration Failed, exiting

A quick check with tools like free and df show there is plenty of space on the system.  I’m not out of disk space; this is a new error for me.

As it turns out, Apache has been leaving stuff left behind in shared memory that needs to be cleaned up.  You can use the ipcs -s command to find this.  When Apache is stopped, we simply flush these out with the following incantation.

for s in $(ipcs -s | awk '/^0x.*www\-data/ { print $2 }')
do
    ipcrm -s $s
done

And hooray, now we can restart Apache!

To prevent this error, you may want to go ahead an disable mod_auth_digest, if you are not using it.

You may want to adjust the allocation of shared memory on your system, to allow for more Semaphore Arrays, if you use the auth_digest.  Take a look at AuthDigestShmemSize parameter as well