MAX_ARGS is reached when calling relation-set

[Chris Glass]

While configuring a charm via relation-set (specifically, passing errorfiles to the haproxy charm along with other key/value pairs), we hit the system's MAX_ARGS limit.

There is currently no way to pass an arbitrary quantity of data to relation-set.

A possible solution would include adding a flag pointing to a file where a prepared set of key/value pairs can be read from (realtion-set --file <path to key-value file>), but of course that is only a suggestion.

[Chris Glass (tribaal)]

Juju/agent version is 1.22.0.1

[Chris Glass (tribaal)]

The specific error can be seen here: https://pastebin.canonical.com/128541/

The subprocess call is charm-helpers calling "realation-set" with the long list of key-values, and this reaches MAX_ARGS (on this particular system it's 2048 bytes (one fourth of the stack size, as obtained with "ulimit -s")

[Chris Glass (tribaal)]

Sorry, here's the ubuntu pastebin version of the above, I keep forgetting to switch between those... http://pastebin.ubuntu.com/10689502/

[Mark Ramm (mark-ramm)]

Updating the priority based on today's discussion on the Juju team call.

[Eric Snow (ericsnowcurrently)]

Is this an issue with relation-set or with Python's subprocess or with the shell? Sounds like the last one but indirectly the first (since relation-get is called via the shell). In that case, I agree that a new option on relation-set to take a file would solve the problem.

How often is this a problem? How many key-value pairs are we talking here?

[Chris Glass (tribaal)]

So, it's an OS-level limitation (it's compile-time for the kernel):
The size of the command-line arguments plus environment variables cannot exceed 1/4th of the stack (ulimit -s), on the default ubuntu kernel.

The usual solution to this problem in command-line programs is to either pass data as files or through stdin, but the relation-set command accepts neither.

Example:
The HAproxy charm takes error files as relation parameters, and given the distributed nature of the problem expects these files to be passed b64encoded (file1=b64encoded(...) file2=b64encoded(...)). So it's not really surprising to exceed MAX_ARGS in this case (several static files can easily exceed MAX_ARGS on containers, for example).

[Eric Snow (ericsnowcurrently)]

http://reviews.vapour.ws/r/1380/

[Eric Snow (ericsnowcurrently)]

Here's another bug that could probably use the same solution: lp:1274460. I'm sure that applies to other commands as well, including the new status-set uniter command.

[Eric Snow (ericsnowcurrently)]

FYI, the --file arg must now be a YAML file containing a map of key/value pairs. If "-" is used for --file then the file is read from stdin.