It's not hardcoded. Its a limit in different places.
On the 32-bit buildds, no one process can address that much memory anyway.
On the 64-bit buildds, many of them don't have (or didn't when I left
:P - the situation may have changed) enough RAM that having a large
swap partition makes sense - if a build thrashes too much, it will
timeout and be killed. (So the rule is 'builds must complete without
causing the build machine to thrash').
Now, for *some* of the builder machines we could run larger VMs with
8GB of RAM, but we have no way in the current software of restricting
the scheduling builds onto them, so shogun would simply end up failing
much of the time when it scheduled onto smaller VMs.
Could we discard all the smaller machines and just run a smaller set
of such larger VMs? Sure. But that then ends up wasting physical ram
for most of the VMs most of the time.
I accept that there is a definite issue for your sofware. However,
there isn't a code bug here AFAICT. I suggest filing a support ticket
if you want to take it further in the operational direction.
That said, even for a scientific operation, various things we've
learnt about software maintenance - such as having a single clear
focus for a module - will still apply, won't they ? Building a mismash
of other projects and doing - I presume - a static link [why else
would you include webkit in your build?] - is going to pose
maintenance and and robustness issues. It is of course your perogative
to accept your build footprint as a given rather than something that
can be addressed.
It's not hardcoded. Its a limit in different places.
On the 32-bit buildds, no one process can address that much memory anyway.
On the 64-bit buildds, many of them don't have (or didn't when I left
:P - the situation may have changed) enough RAM that having a large
swap partition makes sense - if a build thrashes too much, it will
timeout and be killed. (So the rule is 'builds must complete without
causing the build machine to thrash').
Now, for *some* of the builder machines we could run larger VMs with
8GB of RAM, but we have no way in the current software of restricting
the scheduling builds onto them, so shogun would simply end up failing
much of the time when it scheduled onto smaller VMs.
Could we discard all the smaller machines and just run a smaller set
of such larger VMs? Sure. But that then ends up wasting physical ram
for most of the VMs most of the time.
I accept that there is a definite issue for your sofware. However,
there isn't a code bug here AFAICT. I suggest filing a support ticket
if you want to take it further in the operational direction.
That said, even for a scientific operation, various things we've
learnt about software maintenance - such as having a single clear
focus for a module - will still apply, won't they ? Building a mismash
of other projects and doing - I presume - a static link [why else
would you include webkit in your build?] - is going to pose
maintenance and and robustness issues. It is of course your perogative
to accept your build footprint as a given rather than something that
can be addressed.