Activity log for bug #1420057

Date Who What changed Old value New value Message
2015-02-10 01:28:13 Menno Finlay-Smits bug added bug
2015-02-10 01:28:13 Menno Finlay-Smits attachment added Example machine agent log demonstrating the issue https://bugs.launchpad.net/bugs/1420057/+attachment/4315873/+files/machine.log.bz2
2015-02-10 01:29:37 Menno Finlay-Smits description While investigating a customer OpenStack deployment managed by Juju I noticed that many unit and machine agents were failing due to file handle exhaustion ("too many open files") after many failed connections to the (broken) Juju state servers. These agents weren't able to reconnect until they were manually restarted. My guess is that a failed API connection attempt is leaks file handles, possibly a socket (but this is just a guess at this stage). It looks like it took about 2 days of failed connection attempts before file handles were exhausted. The issue was seen with Juju 1.20.9 but it is likely that it's still there in more recent versions. While investigating a customer OpenStack deployment managed by Juju I noticed that many unit and machine agents were failing due to file handle exhaustion ("too many open files") after many failed connections to the (broken) Juju state servers. These agents weren't able to reconnect until they were manually restarted. My guess is that a failed API connection attempt leaks at least one file handle (but this is just a guess at this stage). It looks like it took about 2 days of failed connection attempts before file handles were exhausted. The issue was seen with Juju 1.20.9 but it is likely that it's still there in more recent versions.
2015-02-10 01:36:30 Tim Penhey juju-core: status New Triaged
2015-02-10 14:27:41 Curtis Hovey juju-core: milestone 1.21.3
2015-02-10 15:37:07 Curtis Hovey juju-core: milestone 1.21.3 1.23
2015-02-10 15:37:14 Curtis Hovey nominated for series juju-core/1.21
2015-02-10 15:37:14 Curtis Hovey bug task added juju-core/1.21
2015-02-10 15:37:14 Curtis Hovey nominated for series juju-core/1.22
2015-02-10 15:37:14 Curtis Hovey bug task added juju-core/1.22
2015-02-10 15:37:21 Curtis Hovey juju-core/1.21: status New Triaged
2015-02-10 15:37:23 Curtis Hovey juju-core/1.22: status New Triaged
2015-02-10 15:37:26 Curtis Hovey juju-core/1.21: importance Undecided High
2015-02-10 15:37:28 Curtis Hovey juju-core/1.22: importance Undecided High
2015-02-10 15:37:31 Curtis Hovey juju-core/1.21: milestone 1.23
2015-02-10 15:37:36 Curtis Hovey juju-core/1.21: milestone 1.23 1.21.3
2015-02-10 15:37:40 Curtis Hovey juju-core/1.22: milestone 1.22-beta3
2015-02-10 21:38:54 Curtis Hovey juju-core/1.22: milestone 1.22-beta3 1.22-beta4
2015-02-10 22:53:35 Curtis Hovey bug task deleted juju-core/1.21
2015-02-10 22:53:41 Curtis Hovey bug task deleted juju-core/1.22
2015-04-21 14:27:18 Curtis Hovey juju-core: milestone 1.23 1.24-alpha1
2015-04-27 21:52:54 Curtis Hovey juju-core: milestone 1.24-alpha1 1.24.0
2015-04-27 23:07:38 Curtis Hovey nominated for series juju-core/1.24
2015-04-27 23:07:38 Curtis Hovey bug task added juju-core/1.24
2015-04-27 23:07:43 Curtis Hovey juju-core/1.24: milestone 1.24.0
2015-04-27 23:07:45 Curtis Hovey juju-core: milestone 1.24.0 1.25.0
2015-04-27 23:07:47 Curtis Hovey juju-core/1.24: importance Undecided High
2015-04-27 23:07:50 Curtis Hovey juju-core/1.24: status New Triaged
2015-05-13 17:19:59 Nate Finch tags cpce
2015-05-13 17:20:27 Nate Finch tags cpce
2015-05-14 01:05:32 Cheryl Jennings juju-core: assignee Cheryl Jennings (cherylj)
2015-05-14 01:05:36 Cheryl Jennings juju-core/1.24: assignee Cheryl Jennings (cherylj)
2015-05-14 10:37:34 Katherine Cox-Buday juju-core/1.24: importance High Critical
2015-05-18 02:13:56 Dave Cheney nominated for series juju-core/1.22
2015-05-18 02:13:56 Dave Cheney bug task added juju-core/1.22
2015-05-18 02:13:56 Dave Cheney nominated for series juju-core/1.23
2015-05-18 02:13:56 Dave Cheney bug task added juju-core/1.23
2015-05-18 02:14:04 Dave Cheney juju-core/1.22: assignee Dave Cheney (dave-cheney)
2015-05-18 02:14:07 Dave Cheney juju-core/1.23: assignee Dave Cheney (dave-cheney)
2015-05-18 02:14:17 Dave Cheney juju-core/1.22: status New In Progress
2015-05-18 02:14:20 Dave Cheney juju-core/1.23: status New In Progress
2015-05-18 02:14:23 Dave Cheney juju-core/1.24: status Triaged In Progress
2015-05-18 02:14:27 Dave Cheney juju-core/1.22: importance Undecided Critical
2015-05-18 02:14:30 Dave Cheney juju-core/1.23: importance Undecided Critical
2015-05-18 02:15:00 Dave Cheney juju-core/1.24: assignee Cheryl Jennings (cherylj) Dave Cheney (dave-cheney)
2015-05-18 02:15:09 Dave Cheney juju-core: assignee Cheryl Jennings (cherylj) Dave Cheney (dave-cheney)
2015-05-18 03:19:20 Dave Cheney juju-core/1.22: status In Progress Invalid
2015-05-18 03:30:47 Dave Cheney juju-core/1.22: status Invalid In Progress
2015-05-18 03:30:50 Dave Cheney juju-core: status Triaged Fix Committed
2015-05-18 04:02:19 John A Meinel juju-core/1.22: milestone 1.22.4
2015-05-18 05:44:56 Dave Cheney juju-core/1.23: status In Progress Fix Committed
2015-05-18 05:45:03 Dave Cheney juju-core/1.23: milestone 1.23.4
2015-05-18 06:03:21 Dave Cheney juju-core/1.24: status In Progress Fix Committed
2015-05-18 08:21:48 Dave Cheney juju-core/1.22: status In Progress Fix Committed
2015-05-18 21:24:07 Curtis Hovey juju-core/1.24: milestone 1.24.0 1.24-beta4
2015-05-19 23:19:56 Curtis Hovey juju-core/1.24: status Fix Committed Fix Released
2015-05-20 02:43:22 Curtis Hovey juju-core: status Fix Committed Fix Released
2015-05-28 16:39:59 Curtis Hovey juju-core/1.22: status Fix Committed Fix Released
2016-11-17 00:33:36 Anastasia juju-core/1.23: status Fix Committed Fix Released