Removing a container does not remove the underlying MAAS device representing the container unless the host is also removed.

Bug #1585878 reported by Dongwon Cho
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Fix Released
High
Christian Muirhead

Bug Description

It does not remove the DHCP and DNS configurations on MAAS after juju removes LXD container(machine).

dpkg -l '*maas*' | cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===============================-====================================-============-=================================================
ii maas 2.0.0~beta5+bzr5026-0ubuntu1~xenial1 all "Metal as a Service" is a physical cloud and IPAM
ii maas-cli 2.0.0~beta5+bzr5026-0ubuntu1~xenial1 all MAAS client and command-line interface
un maas-cluster-controller <none> <none> (no description available)
ii maas-common 2.0.0~beta5+bzr5026-0ubuntu1~xenial1 all MAAS server common files
ii maas-dhcp 2.0.0~beta5+bzr5026-0ubuntu1~xenial1 all MAAS DHCP server
ii maas-dns 2.0.0~beta5+bzr5026-0ubuntu1~xenial1 all MAAS DNS server
ii maas-proxy 2.0.0~beta5+bzr5026-0ubuntu1~xenial1 all MAAS Caching Proxy
ii maas-rack-controller 2.0.0~beta5+bzr5026-0ubuntu1~xenial1 all Rack Controller for MAAS
ii maas-region-api 2.0.0~beta5+bzr5026-0ubuntu1~xenial1 all Region controller API service for MAAS
ii maas-region-controller 2.0.0~beta5+bzr5026-0ubuntu1~xenial1 all Region Controller for MAAS
un maas-region-controller-min <none> <none> (no description available)
un python-django-maas <none> <none> (no description available)
un python-maas-client <none> <none> (no description available)
un python-maas-provisioningserver <none> <none> (no description available)
ii python3-django-maas 2.0.0~beta5+bzr5026-0ubuntu1~xenial1 all MAAS server Django web framework (Python 3)
ii python3-maas-client 2.0.0~beta5+bzr5026-0ubuntu1~xenial1 all MAAS python API client (Python 3)
ii python3-maas-provisioningserver 2.0.0~beta5+bzr5026-0ubuntu1~xenial1 all MAAS server provisioning libraries (Python 3)

dpkg -l '*juju*' | cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-================================-============-============================================
ii juju 2.0-beta7-0ubuntu1~16.04.2~juju1 all next generation service orchestration system
ii juju-2.0 2.0-beta7-0ubuntu1~16.04.2~juju1 amd64 Juju is devops distilled - client
un juju-core <none> <none> (no description available)
un juju2 <none> <none> (no description available)

juju add-machine --constraints spaces=mgmt,^public-api,^tunnel,^storage lxd:0

juju show-machine
model: default
machines:
  "0":
    juju-status:
      current: started
      since: 25 May 2016 18:30:08+09:00
      version: 2.0-beta7
    dns-name: 10.192.228.52
    instance-id: 4y3h7p
    machine-status:
      current: running
      message: Deployed
      since: 25 May 2016 18:28:54+09:00
    series: xenial
    containers:
      0/lxd/3:
        juju-status:
          current: started
          since: 26 May 2016 11:19:10+09:00
          version: 2.0-beta7
        dns-name: 10.192.228.56
        instance-id: juju-machine-0-lxd-3
        machine-status:
          current: running
          message: Container started
          since: 26 May 2016 11:18:19+09:00
        series: xenial
    hardware: arch=amd64 cpu-cores=24 mem=128825M availability-zone=default

# On the physical node
sudo lxc list
+----------------------+---------+--------------------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+----------------------+---------+--------------------------------+------+------------+-----------+
| juju-machine-0-lxd-3 | RUNNING | 10.192.228.56 (eth0) | | PERSISTENT | 0 |
| | | 192.168.101.56 (eth1) | | | |
| | | 192.168.102.56 (eth2) | | | |
| | | 10.192.229.56 (eth3) | | | |
+----------------------+---------+--------------------------------+------+------------+-----------+

juju remove-machine 0/lxd/3

juju show-machine
model: default
machines:
  "0":
    juju-status:
      current: started
      since: 25 May 2016 18:30:08+09:00
      version: 2.0-beta7
    dns-name: 10.192.228.52
    instance-id: 4y3h7p
    machine-status:
      current: running
      message: Deployed
      since: 25 May 2016 18:28:54+09:00
    series: xenial
    hardware: arch=amd64 cpu-cores=24 mem=128825M availability-zone=default

dig -x 10.192.228.56

; <<>> DiG 9.10.3-P4-Ubuntu <<>> -x 10.192.228.56
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 59606
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 1, ADDITIONAL: 2

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;56.228.192.10.in-addr.arpa. IN PTR

;; ANSWER SECTION:
56.228.192.10.in-addr.arpa. 30 IN PTR unmordant-shandra.rnd.example.net.
56.228.192.10.in-addr.arpa. 30 IN PTR eth0.unmordant-shandra.rnd.example.net.

;; AUTHORITY SECTION:
228.192.10.in-addr.arpa. 30 IN NS rnd.example.net.

;; ADDITIONAL SECTION:
rnd.example.net. 30 IN A 10.192.228.2

;; Query time: 0 msec
;; SERVER: 10.192.228.2#53(10.192.228.2)
;; WHEN: Thu May 26 14:30:31 KST 2016
;; MSG SIZE rcvd: 154

# on the maas node

cat /etc/bind/maas/zone.rnd.example.net | grep 10.192.228.56
unmordant-shandra 30 IN A 10.192.228.56

cat /var/lib/maas/dhcpd.conf
...
...
host 00-16-3e-b0-82-7a {
   #
   # Node DHCP snippets
   #

   hardware ethernet 00:16:3e:b0:82:7a;
   fixed-address 10.192.228.56;
}

Revision history for this message
Dimiter Naydenov (dimitern) wrote :

Juju registers each container into MAAS as a multi-NIC device with a parent set to its host instance. It's up to MAAS to clean up those settings after the device representing the container is gone along with its host.

affects: juju-core → maas
Revision history for this message
Andres Rodriguez (andreserl) wrote :

Can you list all the devices in MAAS, or go to the Machine details page where the container lives and see if the container is listed there? It may (or may not) be the case that Juju is not removing the containe, hence leaving the records there.

Revision history for this message
Dimiter Naydenov (dimitern) wrote :

Actually, looking at the commands mentioned in the description, I can confirm juju does not delete the device when the container is removed but the host remains.

affects: maas → juju-core
Changed in juju-core:
status: New → Triaged
importance: Undecided → High
summary: - It does not remove the DHCP and DNS configurations on MAAS after juju
- removes LXD container(machine).
+ Removing a container does not remove the underlying MAAS device
+ representing the container unless the host is also removed.
Changed in juju-core:
milestone: none → 2.0-beta9
tags: added: hours maas-provider network reliability
Curtis Hovey (sinzui)
Changed in juju-core:
milestone: 2.0-beta9 → 2.0-beta10
Curtis Hovey (sinzui)
Changed in juju-core:
milestone: 2.0-beta10 → 2.0-beta11
Curtis Hovey (sinzui)
Changed in juju-core:
milestone: 2.0-beta11 → 2.0-beta12
tags: added: 2.0
Changed in juju-core:
milestone: 2.0-beta12 → 2.0-beta13
Changed in juju-core:
assignee: nobody → Christian Muirhead (2-xtian)
Curtis Hovey (sinzui)
Changed in juju-core:
milestone: 2.0-beta13 → 2.0-beta14
Changed in juju-core:
status: Triaged → In Progress
Revision history for this message
Christian Muirhead (2-xtian) wrote :

PR to add ReleaseContainerAddresses to the Networking interface and the MAAS provider.
https://github.com/juju/juju/pull/5883

Revision history for this message
Christian Muirhead (2-xtian) wrote :

PR with state methods to be used by the machine removal worker: https://github.com/juju/juju/pull/5923

Revision history for this message
Christian Muirhead (2-xtian) wrote :

PR to expose the API for the machine undertaker: https://github.com/juju/juju/pull/5924

Curtis Hovey (sinzui)
Changed in juju-core:
milestone: 2.0-beta14 → 2.0-beta15
Changed in juju-core:
milestone: 2.0-beta15 → 2.0-beta16
affects: juju-core → juju
Changed in juju:
milestone: 2.0-beta16 → none
milestone: none → 2.0-beta16
Revision history for this message
Christian Muirhead (2-xtian) wrote :

More PRs:

Simplify ReleaseContainerAddresses interface by requiring less information.
https://github.com/juju/juju/pull/5979

Expose machine.MarkForRemoval in provisioner API
https://github.com/juju/juju/pull/6042

Client side API for machine undertaker
https://github.com/juju/juju/pull/6053

Handle duplicate devices coming back from MAAS in ReleaseContainerAddresses
https://github.com/juju/juju/pull/6080

Enable multiple MAC addresses on test devices in gomaasapi
https://github.com/juju/gomaasapi/pull/56

Final one to add the machine undertaker worker (just finishing off the unit tests)
https://github.com/juju/juju/pull/6082

Curtis Hovey (sinzui)
Changed in juju:
milestone: 2.0-beta16 → 2.0-beta17
Revision history for this message
Christian Muirhead (2-xtian) wrote :

The unit tests are done on the last PR now, waiting for review.

Changed in juju:
status: In Progress → Fix Committed
Curtis Hovey (sinzui)
Changed in juju:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.