Database WAL PENDING_CAP should be configurable

[clayg]

Our databases use a hand rolled write ahead log because everytime we'd tried to benchmark and test the sqlite write ahead log it seemed like it didn't flush quite the way we wanted to - or at least it seemed to prefer to flush much less frequently than we would have liked leading to the database being locked for longer than we were comfortable with. Maybe we should revisit that, but in the meantime our .pending file schema works REALLY well for our needs.

But it could still probably be better!

One easy improvement would be make PENDING_CAP configurable.

https://github.com/openstack/swift/blob/master/swift/common/db.py#L54

Currently we flush to the DB anytime we manage to get a lock on database directory and notice the .pending file is larger than 128KiB which was a pretty good middle of the road answer for slow disks with small updates.

Since we've started content-type and meta updates (fast POST) and and allow encryption and extra swift data like in slo manifests AND gotten MUCH faster disks. I'm questioning if it's the BEST value for ever cluster.

I started playing with increasing it by an order of magnitude and I'm not sure it shouldn't be 100 times bigger. So that makes me think it would be worth the effort to allow operators to experiment with this value in their labs and production clusters.