cannot use release API on stuck observed IPs

[Rodrigo Barbieri]

One of the ways that should be possible to workaround bug https://bugs.launchpad.net/maas/+bug/1896292 is to use the release API, either through:

the CLI (maas root ipaddresses release ip='192.168.105.227' force=true discovered=true)

or API POST http://{{maas_url}}/MAAS/api/2.0/ipaddresses/?op=release

However, when using this API to remove IP 192.168.105.227 which is stuck to a machine in Ready state after successful commissioned and modified reserved dynamic range to exclude this IP (as per instructions in #1896292), those are the results (from MaaS 2.7.3-8291-g.384e521e6-0ubuntu1~18.04.2):

maas root ipaddresses release ip='192.168.105.227'

IP address 192.168.105.227 does not exist, is not the correct type of address, or does not belong to the requesting user.
If you are sure you want to release this address, use force=true as a MAAS administrator.

maas root ipaddresses release ip='192.168.105.227' force=true

IP address 192.168.105.227 does not exist.

maas root ipaddresses release ip='192.168.105.227' discovered=true

IP address 192.168.105.227 does not exist, is not the correct type of address, or does not belong to the requesting user.
If you are sure you want to release this address, use force=true as a MAAS administrator.

maas root ipaddresses release ip='192.168.105.227' force=true discovered=true

Unknown alloc_type.

From regiond.log:

2020-10-01 21:04:42 regiond: [info] 127.0.0.1 GET /MAAS/rpc/ HTTP/1.1 --> 200 OK (referrer: -; agent: provisioningserver.rpc.clusterservice.ClusterClientService)
2020-10-01 21:04:46 maasserver: [error] ################################ Exception: Unknown alloc_type. ################################
2020-10-01 21:04:46 maasserver: [error] Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python3/dist-packages/maasserver/utils/views.py", line 291, in view_atomic_with_post_commit_savepoint
    return view_atomic(*args, **kwargs)
  File "/usr/lib/python3.6/contextlib.py", line 52, in inner
    return func(*args, **kwds)
  File "/usr/lib/python3/dist-packages/maasserver/api/support.py", line 57, in __call__
    response = upcall(request, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/django/views/decorators/vary.py", line 21, in inner_func
    response = func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/piston3/resource.py", line 190, in __call__
    result = self.error_handler(e, request, meth, em_format)
  File "/usr/lib/python3/dist-packages/piston3/resource.py", line 188, in __call__
    result = meth(request, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/maasserver/api/support.py", line 313, in dispatch
    return function(self, request, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/maasserver/api/ip_addresses.py", line 307, in release
    interface.unlink_ip_address(ip_address)
  File "/usr/lib/python3/dist-packages/maasserver/models/interface.py", line 1186, in unlink_ip_address
    mode = ip_address.get_interface_link_type()
  File "/usr/lib/python3/dist-packages/maasserver/models/staticipaddress.py", line 884, in get_interface_link_type
    raise ValueError("Unknown alloc_type.")
ValueError: Unknown alloc_type.

2020-10-01 21:04:46 regiond: [info] 127.0.0.1 POST /MAAS/api/2.0/ipaddresses/?op=release HTTP/1.1 --> 500 INTERNAL_SERVER_ERROR (referrer: -; agent: Python-httplib2/0.9.2 (gzip))

[Rodrigo Barbieri (rodrigo-barbieri2010)]

Adding IPADDRESS_TYPE.DISCOVERED to the condition at [0], causes the code to go to [1] when invoked by the release API, resulting in the delete of the IP similarly to how it is done in [2].

I also tested coding something similar to [3] which does ip=None and then ip.save, but in my testing the result was the same as ip.delete().

[0] https://github.com/maas/maas/blob/04a456af79ebdfd4858bd317f48e42959cc97670/src/maasserver/models/staticipaddress.py#L876

[1] https://github.com/maas/maas/blob/04a456af79ebdfd4858bd317f48e42959cc97670/src/maasserver/models/interface.py#L1189

[2] https://github.com/maas/maas/blob/04a456af79ebdfd4858bd317f48e42959cc97670/src/maasserver/rpc/leases.py#L149

[3] https://github.com/maas/maas/blob/04a456af79ebdfd4858bd317f48e42959cc97670/src/maasserver/rpc/leases.py#L212

[Adam Dyess (addyess)]

Following the logic of this code. It was possible to update the sql database and then delete the ip addresses as a workaround:

$sudo -iu postgres psql
\c maasdb \\ -- switch to maasdb
UPDATE maasserver_staticipaddress SET alloc_type = 5 where ip = '<ip-address>' AND alloc_type = 6;
^D
$ maas root ipaddresses release ip='<ip-address>' force=true

[Igor Gnip (igorgnip)]

Yes, to cleanup accumulated 'observed' IPs:

select id,subnet_id,ip from maasserver_staticipaddress where alloc_type=6 order by ip;
delete from maasserver_dnsresource_ip_addresses where staticipaddress_id in (select id from maasserver_staticipaddress where alloc_type=6 order by ip );
delete from maasserver_interface_ip_addresses where staticipaddress_id in (select id from maasserver_staticipaddress where alloc_type=6 order by ip );
delete from maasserver_staticipaddress where alloc_type=6;

It is still not clear to me how something which was just 'observed' at some point in time maybe a year ago could prevent usage of that ip. It's just observed at some point in time. It's gone. Maas just did not unobserve it.

Un addition to the fix issued which enables deletion via cli/api (thank you), maas could allow alloc_type=6 ips to be overwritten/treated as unused or pinged from rack controller to verify the ip is still 'observed' and if not, remove it.