Comment 10 for bug 1420057

Revision history for this message
Dave Cheney (dave-cheney) wrote : Re: [Bug 1420057] Re: agents see "too many open files" errors after many failed API attempts

I have a fix for 1.22 pending review.

On Mon, May 18, 2015 at 2:04 PM, John A Meinel <email address hidden> wrote:
> Given we saw the bug in 1.20 and 1.22 is the stable release we are
> targeting, I think we still need a fix here. I'm guessing we're using an
> older version of the library before it was 'x/net', can we just bugfix
> that library, or do we have to switch our dependencies in 1.22?
>
> If the dependency fix is too much, we *could* probably get away with
> WontFix on 1.22, as this should only be a problem when an environment is
> unhappy. But we'd definitely want people to know that this bug exists
> and that if they have API servers going down they may need to restart
> all of their Juju agents.
>
>
> ** Changed in: juju-core/1.22
> Milestone: None => 1.22.4
>
> --
> You received this bug notification because you are a bug assignee.
> Matching subscriptions: MOAR JUJU SPAM!
> https://bugs.launchpad.net/bugs/1420057
>
> Title:
> agents see "too many open files" errors after many failed API attempts
>
> Status in juju-core:
> Fix Committed
> Status in juju-core 1.22 series:
> In Progress
> Status in juju-core 1.23 series:
> In Progress
> Status in juju-core 1.24 series:
> In Progress
>
> Bug description:
> While investigating a customer OpenStack deployment managed by Juju I
> noticed that many unit and machine agents were failing due to file
> handle exhaustion ("too many open files") after many failed
> connections to the (broken) Juju state servers. These agents weren't
> able to reconnect until they were manually restarted.
>
> My guess is that a failed API connection attempt leaks at least one
> file handle (but this is just a guess at this stage). It looks like it
> took about 2 days of failed connection attempts before file handles
> were exhausted.
>
> The issue was seen with Juju 1.20.9 but it is likely that it's still
> there in more recent versions.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/juju-core/+bug/1420057/+subscriptions