Activity log for bug #1527068

Date Who What changed Old value New value Message
2015-12-17 06:18:05 Curtis Hovey bug added bug
2015-12-18 18:28:09 Curtis Hovey juju-core: status Triaged Incomplete
2015-12-18 21:52:22 Curtis Hovey juju-core: milestone 1.26-alpha3 2.0-alpha1
2016-01-04 17:53:39 Curtis Hovey description Juju QA's maas is exhausting is static IPs jobs fail because machines and lxc containers cannot be provisioned. This issue has been happening for about 1 day, and all the revision tested have been from master. The mass 1.8 is allocated 80 static IPs, We can see the number used increase with each machine or container deployed. the IPs are not releases when juju destroys the environment. I can see that most calls to destroy use the --force option. Juju QA's maas is exhausting is static IPs jobs fail because machines and lxc containers cannot be provisioned. This issue has been happening for about 1 day, and all the revision tested have been from master. The mass 1.8 is allocated 80 static IPs, We can see the number used increase with each machine or container deployed. the IPs are not releases when juju destroys the environment. I can see that most calls to destroy use the --force option. This is not seen on maas 1.9 and maas 1.7 which are tested with the same scripts.
2016-01-04 18:04:38 Andrew McDermott bug added subscriber Andrew McDermott
2016-01-04 18:19:01 Cheryl Jennings nominated for series juju-core/1.25
2016-01-04 18:19:01 Cheryl Jennings bug task added juju-core/1.25
2016-01-04 18:19:07 Cheryl Jennings juju-core/1.25: status New Triaged
2016-01-04 18:19:14 Cheryl Jennings juju-core/1.25: importance Undecided Critical
2016-01-04 18:19:16 Cheryl Jennings juju-core/1.25: milestone 1.25.2
2016-01-04 22:39:23 Curtis Hovey juju-core: importance Critical High
2016-01-05 03:30:16 John George attachment added maas-1_8-deployer_3430.tgz https://bugs.launchpad.net/juju-core/+bug/1527068/+attachment/4544082/+files/maas-1_8-deployer_3430.tgz
2016-01-05 03:34:40 John George attachment added maas-1_8-deployer_1.25.0_stable.tgz https://bugs.launchpad.net/juju-core/+bug/1527068/+attachment/4544083/+files/maas-1_8-deployer_1.25.0_stable.tgz
2016-01-06 03:26:04 Cheryl Jennings bug task added maas
2016-01-06 03:32:53 Cheryl Jennings summary maas 1.8 static ips not released maas 1.8 static ips not released upon device removal
2016-01-06 19:43:22 Mike Pontillo maas: status New Triaged
2016-01-06 19:44:09 Mike Pontillo summary maas 1.8 static ips not released upon device removal When a parent node is released, MAAS retains its child devices' IP addresses
2016-01-06 19:44:32 Mike Pontillo summary When a parent node is released, MAAS retains its child devices' IP addresses MAAS retains child devices' IP addresses when a parent node is released
2016-01-06 19:44:48 Mike Pontillo maas: importance Undecided Critical
2016-01-06 19:44:49 Mike Pontillo maas: milestone 1.9.0
2016-01-06 19:44:51 Mike Pontillo maas: assignee Mike Pontillo (mpontillo)
2016-01-06 19:45:09 Mike Pontillo maas: status Triaged Fix Released
2016-01-06 19:46:05 Mike Pontillo nominated for series maas/1.8
2016-01-06 19:46:05 Mike Pontillo bug task added maas/1.8
2016-01-06 19:46:05 Mike Pontillo nominated for series maas/1.9
2016-01-06 19:46:05 Mike Pontillo bug task added maas/1.9
2016-01-06 19:46:26 Mike Pontillo bug task deleted maas/1.9
2016-01-06 19:46:36 Mike Pontillo maas/1.8: status New Won't Fix
2016-01-06 19:46:38 Mike Pontillo maas/1.8: importance Undecided Critical
2016-01-06 19:46:43 Mike Pontillo maas/1.8: milestone 1.8.4
2016-01-06 21:34:14 John George description Juju QA's maas is exhausting is static IPs jobs fail because machines and lxc containers cannot be provisioned. This issue has been happening for about 1 day, and all the revision tested have been from master. The mass 1.8 is allocated 80 static IPs, We can see the number used increase with each machine or container deployed. the IPs are not releases when juju destroys the environment. I can see that most calls to destroy use the --force option. This is not seen on maas 1.9 and maas 1.7 which are tested with the same scripts. LXC container IPs are not released when Juju QA tests Juju 1.25.2 and the MAAS version is 1.8.3; even when the --force option is passed to destroy-environment. This leads to IP exaustion in the Juju-QA MAAS 1.8 test setup and all future test jobs fail because machines and LXC containers cannot be provisioned. The issue is not seen with MAAS 1.7, MAAS 1.8.2 or MAAS 1.9, which are tested with the same scripts. These symptoms started occuring around the time Juju 1.25.1 was changed to use MAAS devices for container IPs. The Juju-QA test environment was also updated from MAAS 1.8.2 to 1.8.3. With the 1.25.0 released version of Juju, containers are allocated an IP from the dynamic range, that's been defined in the MAAS cluster config. This is true for both MAAS version 1.8.2 and 1.8.3. Development buids of Juju 1.25.2 tested against MAAS 1.8.2 also get an IP from the dynamic range. The dynamic range IPs are listed in /var/lib/maas/dhcp/dhcpd.leases with a lease expire time defined, similar to the following: lease 10.0.20.17 { starts 3 2016/01/06 20:26:06; ends 4 2016/01/07 08:26:06; cltt 3 2016/01/06 20:26:06; binding state active; next binding state free; rewind binding state free; hardware ethernet 00:16:3e:c2:18:f5; client-hostname "juju-machine-2-lxc-1"; } Development buids of Juju 1.25.2 tested against MAAS 1.8.3 are allocated an IP from the static range, that's been defined in the MAAS cluster config. Entries in /var/lib/maas/dhcp/dhcpd.leases for these IPs do not include a lease expire time, similar to the following: host 10.0.80.117 { dynamic; hardware ethernet 00:16:3e:3a:1e:e1; fixed-address 10.0.80.117; } During testing, an environment is deployed using a bundle, that installs two services into LXC containers. The container IPs can be seen in the Juju status output. Once the environment is destroyed the /var/lib/maas/dhcp/dhcpd.leases file on the MAAS server will include entries for the LXC container IPs, that do not have a correcponding 'deleted' entry. Non-LXC IPs do have corresponding 'deleted' entries, similar to the following: host 10.0.20.111 { dynamic; deleted; }
2016-01-06 21:41:12 John George attachment added juju-status_leases_and_MAAS1.8.2-network-config.tgz https://bugs.launchpad.net/juju-core/+bug/1527068/+attachment/4545172/+files/juju-status_leases_and_MAAS1.8.2-network-config.tgz
2016-01-06 22:30:26 John George description LXC container IPs are not released when Juju QA tests Juju 1.25.2 and the MAAS version is 1.8.3; even when the --force option is passed to destroy-environment. This leads to IP exaustion in the Juju-QA MAAS 1.8 test setup and all future test jobs fail because machines and LXC containers cannot be provisioned. The issue is not seen with MAAS 1.7, MAAS 1.8.2 or MAAS 1.9, which are tested with the same scripts. These symptoms started occuring around the time Juju 1.25.1 was changed to use MAAS devices for container IPs. The Juju-QA test environment was also updated from MAAS 1.8.2 to 1.8.3. With the 1.25.0 released version of Juju, containers are allocated an IP from the dynamic range, that's been defined in the MAAS cluster config. This is true for both MAAS version 1.8.2 and 1.8.3. Development buids of Juju 1.25.2 tested against MAAS 1.8.2 also get an IP from the dynamic range. The dynamic range IPs are listed in /var/lib/maas/dhcp/dhcpd.leases with a lease expire time defined, similar to the following: lease 10.0.20.17 { starts 3 2016/01/06 20:26:06; ends 4 2016/01/07 08:26:06; cltt 3 2016/01/06 20:26:06; binding state active; next binding state free; rewind binding state free; hardware ethernet 00:16:3e:c2:18:f5; client-hostname "juju-machine-2-lxc-1"; } Development buids of Juju 1.25.2 tested against MAAS 1.8.3 are allocated an IP from the static range, that's been defined in the MAAS cluster config. Entries in /var/lib/maas/dhcp/dhcpd.leases for these IPs do not include a lease expire time, similar to the following: host 10.0.80.117 { dynamic; hardware ethernet 00:16:3e:3a:1e:e1; fixed-address 10.0.80.117; } During testing, an environment is deployed using a bundle, that installs two services into LXC containers. The container IPs can be seen in the Juju status output. Once the environment is destroyed the /var/lib/maas/dhcp/dhcpd.leases file on the MAAS server will include entries for the LXC container IPs, that do not have a correcponding 'deleted' entry. Non-LXC IPs do have corresponding 'deleted' entries, similar to the following: host 10.0.20.111 { dynamic; deleted; } LXC container IPs are not released when Juju QA tests Juju 1.25.2 and the MAAS version is 1.8.3; even when the --force option is passed to destroy-environment. This leads to IP exaustion in the Juju-QA MAAS 1.8 test setup and all future test jobs fail because machines and LXC containers cannot be provisioned. The issue is not seen with MAAS 1.7 or MAAS 1.9, which are tested with the same scripts. These symptoms started occuring around the time Juju 1.25.1 was changed to use MAAS devices for container IPs. The Juju-QA test environment was also updated from MAAS 1.8.2 to 1.8.3. With the 1.25.0 released version of Juju, containers are allocated an IP from the dynamic range, that's been defined in the MAAS cluster config. This is true for both MAAS version 1.8.2 and 1.8.3. The dynamic range IPs are listed in /var/lib/maas/dhcp/dhcpd.leases with a lease expire time defined, similar to the following: lease 10.0.20.17 {   starts 3 2016/01/06 20:26:06;   ends 4 2016/01/07 08:26:06;   cltt 3 2016/01/06 20:26:06;   binding state active;   next binding state free;   rewind binding state free;   hardware ethernet 00:16:3e:c2:18:f5;   client-hostname "juju-machine-2-lxc-1"; } Development buids of Juju 1.25.2 tested against MAAS 1.8.3 are allocated an IP from the static range, that's been defined in the MAAS cluster config. Entries in /var/lib/maas/dhcp/dhcpd.leases for these IPs do not include a lease expire time, similar to the following: host 10.0.80.117 {   dynamic;   hardware ethernet 00:16:3e:3a:1e:e1;   fixed-address 10.0.80.117; } During testing, an environment is deployed using a bundle, that installs two services into LXC containers. The container IPs can be seen in the Juju status output. Once the environment is destroyed the /var/lib/maas/dhcp/dhcpd.leases file on the MAAS server will include entries for the LXC container IPs, that do not have a correcponding 'deleted' entry. Non-LXC IPs do have corresponding 'deleted' entries, similar to the following: host 10.0.20.111 {   dynamic;   deleted; }
2016-01-06 22:55:20 John George description LXC container IPs are not released when Juju QA tests Juju 1.25.2 and the MAAS version is 1.8.3; even when the --force option is passed to destroy-environment. This leads to IP exaustion in the Juju-QA MAAS 1.8 test setup and all future test jobs fail because machines and LXC containers cannot be provisioned. The issue is not seen with MAAS 1.7 or MAAS 1.9, which are tested with the same scripts. These symptoms started occuring around the time Juju 1.25.1 was changed to use MAAS devices for container IPs. The Juju-QA test environment was also updated from MAAS 1.8.2 to 1.8.3. With the 1.25.0 released version of Juju, containers are allocated an IP from the dynamic range, that's been defined in the MAAS cluster config. This is true for both MAAS version 1.8.2 and 1.8.3. The dynamic range IPs are listed in /var/lib/maas/dhcp/dhcpd.leases with a lease expire time defined, similar to the following: lease 10.0.20.17 {   starts 3 2016/01/06 20:26:06;   ends 4 2016/01/07 08:26:06;   cltt 3 2016/01/06 20:26:06;   binding state active;   next binding state free;   rewind binding state free;   hardware ethernet 00:16:3e:c2:18:f5;   client-hostname "juju-machine-2-lxc-1"; } Development buids of Juju 1.25.2 tested against MAAS 1.8.3 are allocated an IP from the static range, that's been defined in the MAAS cluster config. Entries in /var/lib/maas/dhcp/dhcpd.leases for these IPs do not include a lease expire time, similar to the following: host 10.0.80.117 {   dynamic;   hardware ethernet 00:16:3e:3a:1e:e1;   fixed-address 10.0.80.117; } During testing, an environment is deployed using a bundle, that installs two services into LXC containers. The container IPs can be seen in the Juju status output. Once the environment is destroyed the /var/lib/maas/dhcp/dhcpd.leases file on the MAAS server will include entries for the LXC container IPs, that do not have a correcponding 'deleted' entry. Non-LXC IPs do have corresponding 'deleted' entries, similar to the following: host 10.0.20.111 {   dynamic;   deleted; } LXC container IPs are not released when Juju QA tests Juju 1.25.2 and the MAAS version is 1.8.3; even when the --force option is passed to destroy-environment. This leads to IP exaustion in the Juju-QA MAAS 1.8 test setup and all future test jobs fail because machines and LXC containers cannot be provisioned. The issue is not seen with MAAS 1.7 or MAAS 1.9, which are tested with the same scripts. These symptoms started occuring around the time Juju 1.25.1 was changed to use MAAS devices for container IPs. The Juju-QA test environment was also updated from MAAS 1.8.2 to 1.8.3. With the 1.25.0 released version of Juju, containers are allocated an IP from the dynamic range, that's been defined in the MAAS cluster config. This is true for both MAAS version 1.8.2 and 1.8.3. The dynamic range IPs are listed in /var/lib/maas/dhcp/dhcpd.leases with a lease expire time defined, similar to the following: lease 10.0.20.17 {   starts 3 2016/01/06 20:26:06;   ends 4 2016/01/07 08:26:06;   cltt 3 2016/01/06 20:26:06;   binding state active;   next binding state free;   rewind binding state free;   hardware ethernet 00:16:3e:c2:18:f5;   client-hostname "juju-machine-2-lxc-1"; } Development buids of Juju 1.25.2 tested against MAAS 1.8 are allocated an IP from the static range, that's been defined in the MAAS cluster config. Entries in /var/lib/maas/dhcp/dhcpd.leases for these IPs do not include a lease expire time, similar to the following: host 10.0.80.117 {   dynamic;   hardware ethernet 00:16:3e:3a:1e:e1;   fixed-address 10.0.80.117; } During testing, an environment is deployed using a bundle, that installs two services into LXC containers. The container IPs can be seen in the Juju status output. Once the environment is destroyed the /var/lib/maas/dhcp/dhcpd.leases file on the MAAS server will include entries for the LXC container IPs, that do not have a correcponding 'deleted' entry. Non-LXC IPs do have corresponding 'deleted' entries, similar to the following: host 10.0.20.111 {   dynamic;   deleted; }
2016-01-06 22:55:59 John George attachment removed juju-status_leases_and_MAAS1.8.2-network-config.tgz https://bugs.launchpad.net/juju-core/+bug/1527068/+attachment/4545172/+files/juju-status_leases_and_MAAS1.8.2-network-config.tgz
2016-01-11 22:00:12 Cheryl Jennings juju-core: status Incomplete Invalid
2016-01-12 16:39:32 Curtis Hovey juju-core/1.25: status Triaged Fix Released
2016-01-20 15:55:03 Curtis Hovey juju-core: milestone 2.0-alpha1