ConflictNovaUsingAttachment: Detach volume from instance using the Compute API (HTTP 409)

[Mehul]

I have Openstack verions: 2023.2 (Bobcat)

Compute Node: Log file /var/log/nova/nova-compute.log

2024-04-09 14:15:34.197 2366 ERROR nova.volume.cinder [None req-4a5c4103-31b9-4564-b41b-0040960836f7 f9ec6818013149cda9c8ba99488a17d7 4f4b6f31aca14ac7bdbc620c042af7a0 - - default default] Delete attachment failed for attachment 33ea6023-bf2f-4a19-a22c-5a38926cb702. Error: ConflictNovaUsingAttachment: Detach volume from instance 39c80abc-1d33-4381-ada9-f6deca1c8939 using the Compute API (HTTP 409) (Request-ID: req-4c65cc97-25f0-450c-9370-4406787a2f8b) Code: 409: cinderclient.exceptions.ClientException: ConflictNovaUsingAttachment: Detach volume from instance 39c80abc-1d33-4381-ada9-f6deca1c8939 using the Compute API (HTTP 409) (Request-ID: req-4c65cc97-25f0-450c-9370-4406787a2f8b)
2024-04-09 14:15:34.224 2366 ERROR oslo_messaging.rpc.server [None req-4a5c4103-31b9-4564-b41b-0040960836f7 f9ec6818013149cda9c8ba99488a17d7 4f4b6f31aca14ac7bdbc620c042af7a0 - - default default] Exception during message handling: cinderclient.exceptions.ClientException: ConflictNovaUsingAttachment: Detach volume from instance 39c80abc-1d33-4381-ada9-f6deca1c8939 using the Compute API (HTTP 409) (Request-ID: req-4c65cc97-25f0-450c-9370-4406787a2f8b)
2024-04-09 14:15:34.224 2366 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2024-04-09 14:15:34.224 2366 ERROR oslo_messaging.rpc.server File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming

[Jadon Naas (jadonn)]

Thank you for submitting the bug! Could you provide more information about your environment and the errors you are seeing? Can you provide the juju status, juju bundle, and (juju and service) logs from cinder and nova will be needed please.

The Charm Guide has information about how to get the kinds of debug information, logs, and other output we need to help us investigate your report:

https://docs.openstack.org/charm-guide/latest/community/software-bug.html#essential-information

[Jadon Naas (jadonn)]

This bug might be related to the issue with Cinder volumes and service tokens fixed in https://bugs.launchpad.net/ubuntu/+source/nova/+bug/2020111.

[Mehul (mehulv)]

Ubuntu Version: Ubuntu 22.04.4 LTS (GNU/Linux 5.15.0-101-generic x86_64)
OpenStack Version: 2023.2 (Bobcat)
Juju: Not installed

1) I have launched a new instance
2) Created a new volume with 7 GB storage
3) Attached to the volume and I can see the volume on instance
Now the error part is
4) Now detach volume
5) Volume status is stuck with "Detaching" and the error is logged in `nova-compute.log`
6) "Detaching" status is stuck even after sometime

nova-compute.log
cinder-volume.log
nova-api.log

You can check the attachment on the log files with the below time frame

From: 2024-04-11 09:33:02
To: 2024-04-11 09:38:43.571

[Alex Kavanagh (ajkavanagh)]

Hi Mehul

> Juju: Not installed

How have you installed OpenStack, please? (e.g. Kolla, Ansible, devstack, something-else)?

If you are not using Juju/charms then this bug would be better targeted at either the upstream/package for nova or cinder? It's just the charm-nova-compute project is for the Juju nova-compute charm as part of the Charmed OpenStack deployment system.

Thanks.

[Mehul (mehulv)]

Hi Alex,
Thanks for your quick response.

I have installed Openstack manually through official documentation only https://docs.openstack.org/install-guide/environment-packages-ubuntu.html

I got your point, May you transfer this issue to the right project will be a great help.

Thanks

[Alex Kavanagh (ajkavanagh)]

Hi Mehul

Thanks for the confirmation in #5; I'm going to mark the charm-nova-compute bug as invalid. As Jadon mentioned in #2 it might be due to the service token mitigations introduced due to solving CVE-2023-2088 - to see if it is related, then https://security.openstack.org/ossa/OSSA-2023-003.html may be worth consulting on how to check your configuration for working with service tokens. If you've already done this, then it may be a different but.

Thanks.