Activity log for bug #2004173

Date Who What changed Old value New value Message
2023-01-30 15:26:13 Aymen Frikha bug added bug
2023-01-30 15:45:39 Aymen Frikha bug added subscriber Liam Young
2023-01-30 15:49:30 Aymen Frikha description Hello, When we enable CIS hardening using cloud-init scripts: https://pastebin.canonical.com/p/92mhhW7DBf/ it breaks the creation of new volumes with luks when need to be created from an image. +--------------------------------+--------------------------------------+ | Field | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2023-01-25T14:34:37.000000 | | description | None | | encrypted | True | | id | 12474116-a517-4ad9-90b2-c864337a2cfc | | migration_status | None | | multiattach | False | | name | test-vol | | os-vol-host-attr:host | cinder@cinder-ceph#cinder-ceph | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 5e87742376a6410383a0df86cb6efa2d | | properties | | | replication_status | None | | size | 5 | | snapshot_id | None | | source_volid | None | | status | error | | type | __DEFAULT__ | | updated_at | 2023-01-25T14:35:29.000000 | | user_id | 5e079b88678645c2b090aee6f53f9f96 | | volume_image_metadata | {'signature_verified': 'False'} | +--------------------------------+--------------------------------------+ steps to test this: openstack volume type create --encryption-provider nova.volume.encryptors.luks.LuksEncryptor --encryption-cipher aes-xts-plain64 --encryption-key-size 256 --encryption-control-location front-end LuksEncryptor-Template-256 openstack volume create --type LuksEncryptor-Template-256 test-vol-bionic-kvm-4 --size 5 --image bionic-kvm Hello, When we enable CIS hardening using cloud-init scripts: https://pastebin.canonical.com/p/753HdJn2QC/ it breaks the creation of new volumes with luks when need to be created from an image. +--------------------------------+--------------------------------------+ | Field | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2023-01-25T14:34:37.000000 | | description | None | | encrypted | True | | id | 12474116-a517-4ad9-90b2-c864337a2cfc | | migration_status | None | | multiattach | False | | name | test-vol | | os-vol-host-attr:host | cinder@cinder-ceph#cinder-ceph | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 5e87742376a6410383a0df86cb6efa2d | | properties | | | replication_status | None | | size | 5 | | snapshot_id | None | | source_volid | None | | status | error | | type | __DEFAULT__ | | updated_at | 2023-01-25T14:35:29.000000 | | user_id | 5e079b88678645c2b090aee6f53f9f96 | | volume_image_metadata | {'signature_verified': 'False'} | +--------------------------------+--------------------------------------+ steps to test this: openstack volume type create --encryption-provider nova.volume.encryptors.luks.LuksEncryptor --encryption-cipher aes-xts-plain64 --encryption-key-size 256 --encryption-control-location front-end LuksEncryptor-Template-256 openstack volume create --type LuksEncryptor-Template-256 test-vol-bionic-kvm-4 --size 5 --image bionic-kvm
2023-01-30 15:49:45 Aymen Frikha information type Private Security Public
2023-01-30 15:51:08 Aymen Frikha description Hello, When we enable CIS hardening using cloud-init scripts: https://pastebin.canonical.com/p/753HdJn2QC/ it breaks the creation of new volumes with luks when need to be created from an image. +--------------------------------+--------------------------------------+ | Field | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2023-01-25T14:34:37.000000 | | description | None | | encrypted | True | | id | 12474116-a517-4ad9-90b2-c864337a2cfc | | migration_status | None | | multiattach | False | | name | test-vol | | os-vol-host-attr:host | cinder@cinder-ceph#cinder-ceph | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 5e87742376a6410383a0df86cb6efa2d | | properties | | | replication_status | None | | size | 5 | | snapshot_id | None | | source_volid | None | | status | error | | type | __DEFAULT__ | | updated_at | 2023-01-25T14:35:29.000000 | | user_id | 5e079b88678645c2b090aee6f53f9f96 | | volume_image_metadata | {'signature_verified': 'False'} | +--------------------------------+--------------------------------------+ steps to test this: openstack volume type create --encryption-provider nova.volume.encryptors.luks.LuksEncryptor --encryption-cipher aes-xts-plain64 --encryption-key-size 256 --encryption-control-location front-end LuksEncryptor-Template-256 openstack volume create --type LuksEncryptor-Template-256 test-vol-bionic-kvm-4 --size 5 --image bionic-kvm Hello, When we enable CIS hardening using cloud-init scripts: https://pastebin.canonical.com/p/753HdJn2QC/ it breaks the creation of new volumes with luks when need to be created from an image. +--------------------------------+--------------------------------------+ | Field | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2023-01-25T14:34:37.000000 | | description | None | | encrypted | True | | id | 12474116-a517-4ad9-90b2-c864337a2cfc | | migration_status | None | | multiattach | False | | name | test-vol | | os-vol-host-attr:host | cinder@cinder-ceph#cinder-ceph | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 5e87742376a6410383a0df86cb6efa2d | | properties | | | replication_status | None | | size | 5 | | snapshot_id | None | | source_volid | None | | status | error | | type | __DEFAULT__ | | updated_at | 2023-01-25T14:35:29.000000 | | user_id | 5e079b88678645c2b090aee6f53f9f96 | | volume_image_metadata | {'signature_verified': 'False'} | +--------------------------------+--------------------------------------+ steps to test this: openstack volume type create --encryption-provider nova.volume.encryptors.luks.LuksEncryptor --encryption-cipher aes-xts-plain64 --encryption-key-size 256 --encryption-control-location front-end LuksEncryptor-Template-256 openstack volume create --type LuksEncryptor-Template-256 test-vol-bionic-kvm-4 --size 5 --image bionic-kvm Here is the issue I see from cinder-volume logs: 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex.stderr) 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server cinder.exception.ImageCopyFailure: Failed to copy image to volume: rbd: error opening /var/lib/cinder/conversion/tmpy88g_2co 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: import failed: (13) Permission denied 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: --new-format is deprecated, use --image-format 2 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: --order is deprecated, use --object-size 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server
2023-01-30 15:53:17 Aymen Frikha description Hello, When we enable CIS hardening using cloud-init scripts: https://pastebin.canonical.com/p/753HdJn2QC/ it breaks the creation of new volumes with luks when need to be created from an image. +--------------------------------+--------------------------------------+ | Field | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2023-01-25T14:34:37.000000 | | description | None | | encrypted | True | | id | 12474116-a517-4ad9-90b2-c864337a2cfc | | migration_status | None | | multiattach | False | | name | test-vol | | os-vol-host-attr:host | cinder@cinder-ceph#cinder-ceph | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 5e87742376a6410383a0df86cb6efa2d | | properties | | | replication_status | None | | size | 5 | | snapshot_id | None | | source_volid | None | | status | error | | type | __DEFAULT__ | | updated_at | 2023-01-25T14:35:29.000000 | | user_id | 5e079b88678645c2b090aee6f53f9f96 | | volume_image_metadata | {'signature_verified': 'False'} | +--------------------------------+--------------------------------------+ steps to test this: openstack volume type create --encryption-provider nova.volume.encryptors.luks.LuksEncryptor --encryption-cipher aes-xts-plain64 --encryption-key-size 256 --encryption-control-location front-end LuksEncryptor-Template-256 openstack volume create --type LuksEncryptor-Template-256 test-vol-bionic-kvm-4 --size 5 --image bionic-kvm Here is the issue I see from cinder-volume logs: 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex.stderr) 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server cinder.exception.ImageCopyFailure: Failed to copy image to volume: rbd: error opening /var/lib/cinder/conversion/tmpy88g_2co 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: import failed: (13) Permission denied 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: --new-format is deprecated, use --image-format 2 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: --order is deprecated, use --object-size 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server Hello, When we enable CIS hardening using cloud-init scripts: https://pastebin.canonical.com/p/753HdJn2QC/ it breaks the creation of new volumes with luks when need to be created from an image. +--------------------------------+--------------------------------------+ | Field | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2023-01-25T14:34:37.000000 | | description | None | | encrypted | True | | id | 12474116-a517-4ad9-90b2-c864337a2cfc | | migration_status | None | | multiattach | False | | name | test-vol | | os-vol-host-attr:host | cinder@cinder-ceph#cinder-ceph | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 5e87742376a6410383a0df86cb6efa2d | | properties | | | replication_status | None | | size | 5 | | snapshot_id | None | | source_volid | None | | status | error | | type | __DEFAULT__ | | updated_at | 2023-01-25T14:35:29.000000 | | user_id | 5e079b88678645c2b090aee6f53f9f96 | | volume_image_metadata | {'signature_verified': 'False'} | +--------------------------------+--------------------------------------+ steps to test this: openstack volume type create --encryption-provider nova.volume.encryptors.luks.LuksEncryptor --encryption-cipher aes-xts-plain64 --encryption-key-size 256 --encryption-control-location front-end LuksEncryptor-Template-256 openstack volume create --type LuksEncryptor-Template-256 test-vol-bionic-kvm-4 --size 5 --image bionic-kvm Here is the issue I see from cinder-volume logs: 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex.stderr) 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server cinder.exception.ImageCopyFailure: Failed to copy image to volume: rbd: error opening /var/lib/cinder/conversion/tmpy88g_2co 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: import failed: (13) Permission denied 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: --new-format is deprecated, use --image-format 2 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: --order is deprecated, use --object-size 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server Release: Focal/Yoga stable
2023-01-30 18:45:52 Liam Young charm-cinder: status New Confirmed
2023-01-30 18:46:05 Liam Young charm-cinder: importance Undecided High
2023-01-31 13:00:23 Liam Young summary CIS hardening breaks luks volumes created from images Creating luks volumes from images using rbd fails if roots umask it 027
2023-01-31 13:37:55 Liam Young description Hello, When we enable CIS hardening using cloud-init scripts: https://pastebin.canonical.com/p/753HdJn2QC/ it breaks the creation of new volumes with luks when need to be created from an image. +--------------------------------+--------------------------------------+ | Field | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2023-01-25T14:34:37.000000 | | description | None | | encrypted | True | | id | 12474116-a517-4ad9-90b2-c864337a2cfc | | migration_status | None | | multiattach | False | | name | test-vol | | os-vol-host-attr:host | cinder@cinder-ceph#cinder-ceph | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 5e87742376a6410383a0df86cb6efa2d | | properties | | | replication_status | None | | size | 5 | | snapshot_id | None | | source_volid | None | | status | error | | type | __DEFAULT__ | | updated_at | 2023-01-25T14:35:29.000000 | | user_id | 5e079b88678645c2b090aee6f53f9f96 | | volume_image_metadata | {'signature_verified': 'False'} | +--------------------------------+--------------------------------------+ steps to test this: openstack volume type create --encryption-provider nova.volume.encryptors.luks.LuksEncryptor --encryption-cipher aes-xts-plain64 --encryption-key-size 256 --encryption-control-location front-end LuksEncryptor-Template-256 openstack volume create --type LuksEncryptor-Template-256 test-vol-bionic-kvm-4 --size 5 --image bionic-kvm Here is the issue I see from cinder-volume logs: 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex.stderr) 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server cinder.exception.ImageCopyFailure: Failed to copy image to volume: rbd: error opening /var/lib/cinder/conversion/tmpy88g_2co 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: import failed: (13) Permission denied 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: --new-format is deprecated, use --image-format 2 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: --order is deprecated, use --object-size 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server rbd: -p [ --pool ] is deprecated, use --dest-pool 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server 2023-01-30 14:45:19.080 1290 ERROR oslo_messaging.rpc.server Release: Focal/Yoga stable When the cinder rbd driver converts the image to luks format it calls image_utils.convert_image with the default for run_as_root which is true. The converted image is owned by the root user with a group ownership of root. The rbd driver then overwrites the orignal file with the new converted file which has the affect of changing its ownership from cinder:cinder to root:root. The rbd driver then attempts to call rbd import as the cinder user but this fails as cinder cannot read the source file. The following error appears in the cinder-volume log: cinder.exception.ImageCopyFailure: Failed to copy image to volume: rbd: error opening /var/lib/cinder/conversion/tmpry2juoj_ Full error log here: https://pastebin.ubuntu.com/p/BMS4vvBJy5/ Reproduce: openstack volume type create \ --encryption-provider nova.volume.encryptors.luks.LuksEncryptor \ --encryption-cipher aes-xts-plain64 \ --encryption-key-size 256 \ --encryption-control-location front-end \ LuksEncryptor-Template-256 openstack volume create \ --type LuksEncryptor-Template-256 \ test-vol-cirros-1 \ --size 5 \ --image cirros This should work. On cinder machine: sed -i 's/^UMASK.*/UMASK 027/' /etc/login.defs openstack volume create \ --type LuksEncryptor-Template-256 \ test-vol-cirros-2 \ --size 5 \ --image cirros this will fail.
2023-01-31 20:35:59 Liam Young summary Creating luks volumes from images using rbd fails if roots umask it 027 Creating luks volumes from images using rbd fails if roots umask is 027