Comment 4 for bug 1680316

Revision history for this message
Yavor Nikolov (yavor-nikolov) wrote :

Hi,

> Can we evenly distribute the load of subjected pbzip2 process at the time of backup among CPU's of all cluster members instead of only using resources of sadmapp3.

That's a question not about pbzip2 but about the job which is running it. It's mostly about details of these processes which are running the tar command. If you know more about them you might be able to split it somehow. E.g. - split the files/streams into smaller chunks, evenly distribute the pieces across all servers, then run that command on each of them. Note: if the files are really big - and if you run pbzip2 with same number of threads, you may still cause high CPU usage (on all servers) - it just may take shorter time to complete.

* One possible idea is to copy/redirect these backup streams to another server (not so sensitive in terms of CPU usage) and run compression there.
* Another thing I already suggested: change these commands to use either pbzip2 with -p option (e.g. -p1 or -p2); or use bzip2 instead of pbzip2.