fallocate fails in test environment

[Constantine Peresypkin]

Functional test TestFile.testFileSizeLimit fails in test environment.
The test tries to allocate a space for several replicas of almost 5Gb file, to check the 5Gb limit.
This test will fail if the test environment has less than 3 * 5Gb of free space.
According to SAIO docs the test env is built with 1Gb volume which is obviously not enough.
From docs:
dd if=/dev/zero of=/srv/swift-disk bs=1024 count=0 seek=1000000
   (modify seek to make a larger or smaller partition)

Why this was working before? In my case it was working because VmWare engine prior to 5.0.0 build-812388 was blatantly ignoring fallocate as a noop.
But the problem has a bigger scope: after testFileSizeLimit fails, most of other tests also fail, they just don't have anough free space to run...

I.e. either we disable fallocate for the test duration or make test partition much bigger.

[gholt (gholt)]

The SAIO was updated to include setting disable_fallocate = true in various config files. Does this work in your situation?

[gholt (gholt)]

Oh, I guess a link to said doc might help: http://docs.openstack.org/developer/swift/development_saio.html

[Constantine Peresypkin (constantine-q)]

Hmm, I suppose it was added quite recently, oh, I see, added by you on August 29th.
Three months ago I have created a script that used SAIO doc. It seems it's obsolete now...
Maybe I can contribute the said script to make the changes more obvious... it's not that intuitive to check for doc changes after each pull. :)

[gholt (gholt)]

No joke. :) I forgot I was the one who updated the doc. I do remember somebody else fixed fallocate to actually work and that it caused me no end of problems as it used over 100G during functional tests!

[Constantine Peresypkin (constantine-q)]

Hmm...It's strange because I remember the problems with fallocate not working on Vmware, for a long time.
I've even made a pure C tests under various VMs some 6 month ago, and confirmed it as a Vmware bug, i.e. it was properly fallocating on Linux host but was noop-ing on guest.

[Constantine Peresypkin (constantine-q)]

Not a bug, but an (obscure) documentation change.

[John Dickinson (notmyname)]

The implementation of testFileSizeLimit can be much better. The current implementation (now in 2016) requires 4x the max_file_size (minus 111 bytes) to be fallocat'able on your drives. Sure, sure, just turn off fallocate, but that just hides the issue, and fallocate should be on in prod, so it should be on in dev (IMO).

testFileSizeLimit should be changed so that the successfully created large objects are deleted right after that part of the test passes.

I ran in to this when testing with an 8GB drive and a 2GB max_file_size.