MAX_ARGS is reached when calling relation-set

Bug #1437366 reported by Chris Glass
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Charm Helpers
Fix Released
Undecided
Björn Tillenius
juju-core
Fix Released
Critical
Eric Snow
1.23
Fix Released
Critical
Eric Snow

Bug Description

While configuring a charm via relation-set (specifically, passing errorfiles to the haproxy charm along with other key/value pairs), we hit the system's MAX_ARGS limit.

There is currently no way to pass an arbitrary quantity of data to relation-set.

A possible solution would include adding a flag pointing to a file where a prepared set of key/value pairs can be read from (realtion-set --file <path to key-value file>), but of course that is only a suggestion.

Related branches

Revision history for this message
Chris Glass (tribaal) wrote :

Juju/agent version is 1.22.0.1

Revision history for this message
Chris Glass (tribaal) wrote :

The specific error can be seen here: https://pastebin.canonical.com/128541/

The subprocess call is charm-helpers calling "realation-set" with the long list of key-values, and this reaches MAX_ARGS (on this particular system it's 2048 bytes (one fourth of the stack size, as obtained with "ulimit -s")

Revision history for this message
Chris Glass (tribaal) wrote :

Sorry, here's the ubuntu pastebin version of the above, I keep forgetting to switch between those... http://pastebin.ubuntu.com/10689502/

Curtis Hovey (sinzui)
tags: added: charm relations
Changed in juju-core:
status: New → Triaged
importance: Undecided → Medium
Revision history for this message
Mark Ramm (mark-ramm) wrote :

Updating the priority based on today's discussion on the Juju team call.

Changed in juju-core:
importance: Medium → Critical
Revision history for this message
Eric Snow (ericsnowcurrently) wrote :

Is this an issue with relation-set or with Python's subprocess or with the shell? Sounds like the last one but indirectly the first (since relation-get is called via the shell). In that case, I agree that a new option on relation-set to take a file would solve the problem.

How often is this a problem? How many key-value pairs are we talking here?

Revision history for this message
Chris Glass (tribaal) wrote :

So, it's an OS-level limitation (it's compile-time for the kernel):
The size of the command-line arguments plus environment variables cannot exceed 1/4th of the stack (ulimit -s), on the default ubuntu kernel.

The usual solution to this problem in command-line programs is to either pass data as files or through stdin, but the relation-set command accepts neither.

Example:
The HAproxy charm takes error files as relation parameters, and given the distributed nature of the problem expects these files to be passed b64encoded (file1=b64encoded(...) file2=b64encoded(...)). So it's not really surprising to exceed MAX_ARGS in this case (several static files can easily exceed MAX_ARGS on containers, for example).

Changed in juju-core:
assignee: nobody → Eric Snow (ericsnowcurrently)
milestone: none → 1.24-alpha1
status: Triaged → In Progress
Revision history for this message
Eric Snow (ericsnowcurrently) wrote :
Revision history for this message
Eric Snow (ericsnowcurrently) wrote :

Here's another bug that could probably use the same solution: lp:1274460. I'm sure that applies to other commands as well, including the new status-set uniter command.

Revision history for this message
Eric Snow (ericsnowcurrently) wrote :

FYI, the --file arg must now be a YAML file containing a map of key/value pairs. If "-" is used for --file then the file is read from stdin.

Changed in juju-core:
status: In Progress → Fix Committed
Curtis Hovey (sinzui)
Changed in juju-core:
status: Fix Committed → Fix Released
Changed in charm-helpers:
status: New → In Progress
assignee: nobody → Björn Tillenius (bjornt)
Changed in charm-helpers:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.