| 2023-08-11 11:30:25 |
Lukas Koenen |
bug |
|
|
added bug |
| 2023-08-11 11:31:16 |
Lukas Koenen |
description |
While deploying manila using the "manila-cephfsganesha-config" heat template the deployment fails during the "get dbus-1 file" task which is part of the "tripleo_cephadm" ansible role due to "no space left on device". This happens during the container creation.
## Observations
Unlike other tasks of the role "get dbus-1 file" uses the default user and not root. The default user has very limited disk space (/dev/mapper/vg-lv_home 1.2G 630M 555M 54% /home) and watching disk space during deployment "/dev/mapper/vg-lv_home" spikes up close to 100% usage which might be the reason for the error.
## Possible Solution
Using "become: true" and therefore the root user in the "get dbus-1 file" tasks fixes the problem.
## Steps to reproduce
Add "/usr/share/openstack-tripleo-heat-templates/environments/manila-cephfsganesha-config.yaml" to the deployment.
## Error
{
"changed": false,
"cmd": [
"podman",
"run",
"--rm",
"--entrypoint=cat",
"os-registry.infra.h-da.cloud:5000/tripleowallabycentos9/daemon:current-ceph",
"/etc/dbus-1/system.d/org.ganesha.nfsd.conf"
],
"delta": "0:00:20.367417",
"end": "2023-06-16 10:56:14.870843",
"msg": "non-zero return code",
"rc": 125,
"start": "2023-06-16 10:55:54.503426",
"stderr": "Trying to pull os-registry.infra.h-da.cloud:5000/tripleowallabycentos9/daemon:current-ceph...\nGetting image source signatures\nCopying blob sha256:bb9e6989315f6f108f7cab8ae301e09c9e41a982d776c25860169efaeb5f7530\nCopying blob sha256:f1ee40d9db4a2bf9b96ea48d6cb45c602a6761650f67dc84bba5a0d2495e845a\nCopying blob sha256:0d557d32f54ebd277fdffbbdf656b90442ee9d8753aec9ebac429eee967f4dee\nCopying blob sha256:6c5de04c936da27e33992af1e54e929f1cb39c8e1473d9d25ed1f1dc2d842fd4\nCopying blob sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\nCopying blob sha256:17facd475902d6709cff908630b59271c7ad18f64c3a1d0143d438c6988504ef\nCopying blob sha256:fbd68c38accbcb670817752f4faa8f3f845774fab06dc0b5e4b8c860f787ecd3\nCopying blob sha256:a57fb466f17b81e5b74877994d78b14170898db92c980ec5842dcfcbeefb324f\nCopying blob sha256:24972a36f4e1f7d9063e3cbc4ff3c171371cdeee9529e5581ab6e352a574b818\nCopying blob sha256:f48748416b3be33005287f0c884e0f98eebd74037311be116f8eff3cd85c3d51\nCopying blob sha256:e6e5b41683139f3942ff86736d3d416020b9b38c7d118df4ccf71e6809136c65\nCopying blob sha256:93cc817d8e84cb7580ee7d1930702f1ec087a93bcb30534a0c23341344113bb4\nCopying blob sha256:6e10a1f887f8a529f097f805ce4f6f64b06f13cfb638a6e1322bebe5113a86f4\nError: writing blob: adding layer with blob \"sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\": processing tar file(write /usr/lib64/samba/libsmbd-base-samba4.so: no space lefton device): exit status 1",
"stderr_lines": [
"Trying to pull os-registry.infra.h-da.cloud:5000/tripleowallabycentos9/daemon:current-ceph...",
"Getting image source signatures",
"Copying blob sha256:bb9e6989315f6f108f7cab8ae301e09c9e41a982d776c25860169efaeb5f7530",
"Copying blob sha256:f1ee40d9db4a2bf9b96ea48d6cb45c602a6761650f67dc84bba5a0d2495e845a",
"Copying blob sha256:0d557d32f54ebd277fdffbbdf656b90442ee9d8753aec9ebac429eee967f4dee",
"Copying blob sha256:6c5de04c936da27e33992af1e54e929f1cb39c8e1473d9d25ed1f1dc2d842fd4",
"Copying blob sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5",
"Copying blob sha256:17facd475902d6709cff908630b59271c7ad18f64c3a1d0143d438c6988504ef",
"Copying blob sha256:fbd68c38accbcb670817752f4faa8f3f845774fab06dc0b5e4b8c860f787ecd3",
"Copying blob sha256:a57fb466f17b81e5b74877994d78b14170898db92c980ec5842dcfcbeefb324f",
"Copying blob sha256:24972a36f4e1f7d9063e3cbc4ff3c171371cdeee9529e5581ab6e352a574b818",
"Copying blob sha256:f48748416b3be33005287f0c884e0f98eebd74037311be116f8eff3cd85c3d51",
"Copying blob sha256:e6e5b41683139f3942ff86736d3d416020b9b38c7d118df4ccf71e6809136c65",
"Copying blob sha256:93cc817d8e84cb7580ee7d1930702f1ec087a93bcb30534a0c23341344113bb4",
"Copying blob sha256:6e10a1f887f8a529f097f805ce4f6f64b06f13cfb638a6e1322bebe5113a86f4",
"Error:writing blob: adding layer with blob \"sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\": processing tar file(write /usr/lib64/samba/libsmbd-base-samba4.so: no space left on device): exit status 1"
],
"stdout": "",
"stdout_lines": []
} |
While deploying manila using the "manila-cephfsganesha-config" heat template the deployment fails during the "get dbus-1 file" task which is part of the "tripleo_cephadm" ansible role (tasks/nfs.yaml) due to "no space left on device". This happens during the container creation.
## Observations
Unlike other tasks of the role "get dbus-1 file" uses the default user and not root. The default user has very limited disk space (/dev/mapper/vg-lv_home 1.2G 630M 555M 54% /home) and watching disk space during deployment "/dev/mapper/vg-lv_home" spikes up close to 100% usage which might be the reason for the error.
## Possible Solution
Using "become: true" and therefore the root user in the "get dbus-1 file" tasks fixes the problem.
## Steps to reproduce
Add "/usr/share/openstack-tripleo-heat-templates/environments/manila-cephfsganesha-config.yaml" to the deployment.
## Error
{
"changed": false,
"cmd": [
"podman",
"run",
"--rm",
"--entrypoint=cat",
"os-registry.infra.h-da.cloud:5000/tripleowallabycentos9/daemon:current-ceph",
"/etc/dbus-1/system.d/org.ganesha.nfsd.conf"
],
"delta": "0:00:20.367417",
"end": "2023-06-16 10:56:14.870843",
"msg": "non-zero return code",
"rc": 125,
"start": "2023-06-16 10:55:54.503426",
"stderr": "Trying to pull os-registry.infra.h-da.cloud:5000/tripleowallabycentos9/daemon:current-ceph...\nGetting image source signatures\nCopying blob sha256:bb9e6989315f6f108f7cab8ae301e09c9e41a982d776c25860169efaeb5f7530\nCopying blob sha256:f1ee40d9db4a2bf9b96ea48d6cb45c602a6761650f67dc84bba5a0d2495e845a\nCopying blob sha256:0d557d32f54ebd277fdffbbdf656b90442ee9d8753aec9ebac429eee967f4dee\nCopying blob sha256:6c5de04c936da27e33992af1e54e929f1cb39c8e1473d9d25ed1f1dc2d842fd4\nCopying blob sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\nCopying blob sha256:17facd475902d6709cff908630b59271c7ad18f64c3a1d0143d438c6988504ef\nCopying blob sha256:fbd68c38accbcb670817752f4faa8f3f845774fab06dc0b5e4b8c860f787ecd3\nCopying blob sha256:a57fb466f17b81e5b74877994d78b14170898db92c980ec5842dcfcbeefb324f\nCopying blob sha256:24972a36f4e1f7d9063e3cbc4ff3c171371cdeee9529e5581ab6e352a574b818\nCopying blob sha256:f48748416b3be33005287f0c884e0f98eebd74037311be116f8eff3cd85c3d51\nCopying blob sha256:e6e5b41683139f3942ff86736d3d416020b9b38c7d118df4ccf71e6809136c65\nCopying blob sha256:93cc817d8e84cb7580ee7d1930702f1ec087a93bcb30534a0c23341344113bb4\nCopying blob sha256:6e10a1f887f8a529f097f805ce4f6f64b06f13cfb638a6e1322bebe5113a86f4\nError: writing blob: adding layer with blob \"sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\": processing tar file(write /usr/lib64/samba/libsmbd-base-samba4.so: no space lefton device): exit status 1",
"stderr_lines": [
"Trying to pull os-registry.infra.h-da.cloud:5000/tripleowallabycentos9/daemon:current-ceph...",
"Getting image source signatures",
"Copying blob sha256:bb9e6989315f6f108f7cab8ae301e09c9e41a982d776c25860169efaeb5f7530",
"Copying blob sha256:f1ee40d9db4a2bf9b96ea48d6cb45c602a6761650f67dc84bba5a0d2495e845a",
"Copying blob sha256:0d557d32f54ebd277fdffbbdf656b90442ee9d8753aec9ebac429eee967f4dee",
"Copying blob sha256:6c5de04c936da27e33992af1e54e929f1cb39c8e1473d9d25ed1f1dc2d842fd4",
"Copying blob sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5",
"Copying blob sha256:17facd475902d6709cff908630b59271c7ad18f64c3a1d0143d438c6988504ef",
"Copying blob sha256:fbd68c38accbcb670817752f4faa8f3f845774fab06dc0b5e4b8c860f787ecd3",
"Copying blob sha256:a57fb466f17b81e5b74877994d78b14170898db92c980ec5842dcfcbeefb324f",
"Copying blob sha256:24972a36f4e1f7d9063e3cbc4ff3c171371cdeee9529e5581ab6e352a574b818",
"Copying blob sha256:f48748416b3be33005287f0c884e0f98eebd74037311be116f8eff3cd85c3d51",
"Copying blob sha256:e6e5b41683139f3942ff86736d3d416020b9b38c7d118df4ccf71e6809136c65",
"Copying blob sha256:93cc817d8e84cb7580ee7d1930702f1ec087a93bcb30534a0c23341344113bb4",
"Copying blob sha256:6e10a1f887f8a529f097f805ce4f6f64b06f13cfb638a6e1322bebe5113a86f4",
"Error:writing blob: adding layer with blob \"sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\": processing tar file(write /usr/lib64/samba/libsmbd-base-samba4.so: no space left on device): exit status 1"
],
"stdout": "",
"stdout_lines": []
} |
|
| 2023-08-11 11:33:34 |
Lukas Koenen |
description |
While deploying manila using the "manila-cephfsganesha-config" heat template the deployment fails during the "get dbus-1 file" task which is part of the "tripleo_cephadm" ansible role (tasks/nfs.yaml) due to "no space left on device". This happens during the container creation.
## Observations
Unlike other tasks of the role "get dbus-1 file" uses the default user and not root. The default user has very limited disk space (/dev/mapper/vg-lv_home 1.2G 630M 555M 54% /home) and watching disk space during deployment "/dev/mapper/vg-lv_home" spikes up close to 100% usage which might be the reason for the error.
## Possible Solution
Using "become: true" and therefore the root user in the "get dbus-1 file" tasks fixes the problem.
## Steps to reproduce
Add "/usr/share/openstack-tripleo-heat-templates/environments/manila-cephfsganesha-config.yaml" to the deployment.
## Error
{
"changed": false,
"cmd": [
"podman",
"run",
"--rm",
"--entrypoint=cat",
"os-registry.infra.h-da.cloud:5000/tripleowallabycentos9/daemon:current-ceph",
"/etc/dbus-1/system.d/org.ganesha.nfsd.conf"
],
"delta": "0:00:20.367417",
"end": "2023-06-16 10:56:14.870843",
"msg": "non-zero return code",
"rc": 125,
"start": "2023-06-16 10:55:54.503426",
"stderr": "Trying to pull os-registry.infra.h-da.cloud:5000/tripleowallabycentos9/daemon:current-ceph...\nGetting image source signatures\nCopying blob sha256:bb9e6989315f6f108f7cab8ae301e09c9e41a982d776c25860169efaeb5f7530\nCopying blob sha256:f1ee40d9db4a2bf9b96ea48d6cb45c602a6761650f67dc84bba5a0d2495e845a\nCopying blob sha256:0d557d32f54ebd277fdffbbdf656b90442ee9d8753aec9ebac429eee967f4dee\nCopying blob sha256:6c5de04c936da27e33992af1e54e929f1cb39c8e1473d9d25ed1f1dc2d842fd4\nCopying blob sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\nCopying blob sha256:17facd475902d6709cff908630b59271c7ad18f64c3a1d0143d438c6988504ef\nCopying blob sha256:fbd68c38accbcb670817752f4faa8f3f845774fab06dc0b5e4b8c860f787ecd3\nCopying blob sha256:a57fb466f17b81e5b74877994d78b14170898db92c980ec5842dcfcbeefb324f\nCopying blob sha256:24972a36f4e1f7d9063e3cbc4ff3c171371cdeee9529e5581ab6e352a574b818\nCopying blob sha256:f48748416b3be33005287f0c884e0f98eebd74037311be116f8eff3cd85c3d51\nCopying blob sha256:e6e5b41683139f3942ff86736d3d416020b9b38c7d118df4ccf71e6809136c65\nCopying blob sha256:93cc817d8e84cb7580ee7d1930702f1ec087a93bcb30534a0c23341344113bb4\nCopying blob sha256:6e10a1f887f8a529f097f805ce4f6f64b06f13cfb638a6e1322bebe5113a86f4\nError: writing blob: adding layer with blob \"sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\": processing tar file(write /usr/lib64/samba/libsmbd-base-samba4.so: no space lefton device): exit status 1",
"stderr_lines": [
"Trying to pull os-registry.infra.h-da.cloud:5000/tripleowallabycentos9/daemon:current-ceph...",
"Getting image source signatures",
"Copying blob sha256:bb9e6989315f6f108f7cab8ae301e09c9e41a982d776c25860169efaeb5f7530",
"Copying blob sha256:f1ee40d9db4a2bf9b96ea48d6cb45c602a6761650f67dc84bba5a0d2495e845a",
"Copying blob sha256:0d557d32f54ebd277fdffbbdf656b90442ee9d8753aec9ebac429eee967f4dee",
"Copying blob sha256:6c5de04c936da27e33992af1e54e929f1cb39c8e1473d9d25ed1f1dc2d842fd4",
"Copying blob sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5",
"Copying blob sha256:17facd475902d6709cff908630b59271c7ad18f64c3a1d0143d438c6988504ef",
"Copying blob sha256:fbd68c38accbcb670817752f4faa8f3f845774fab06dc0b5e4b8c860f787ecd3",
"Copying blob sha256:a57fb466f17b81e5b74877994d78b14170898db92c980ec5842dcfcbeefb324f",
"Copying blob sha256:24972a36f4e1f7d9063e3cbc4ff3c171371cdeee9529e5581ab6e352a574b818",
"Copying blob sha256:f48748416b3be33005287f0c884e0f98eebd74037311be116f8eff3cd85c3d51",
"Copying blob sha256:e6e5b41683139f3942ff86736d3d416020b9b38c7d118df4ccf71e6809136c65",
"Copying blob sha256:93cc817d8e84cb7580ee7d1930702f1ec087a93bcb30534a0c23341344113bb4",
"Copying blob sha256:6e10a1f887f8a529f097f805ce4f6f64b06f13cfb638a6e1322bebe5113a86f4",
"Error:writing blob: adding layer with blob \"sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\": processing tar file(write /usr/lib64/samba/libsmbd-base-samba4.so: no space left on device): exit status 1"
],
"stdout": "",
"stdout_lines": []
} |
While deploying manila using the "manila-cephfsganesha-config" heat template the deployment fails during the "get dbus-1 file" task which is part of the "tripleo_cephadm" ansible role (tasks/nfs.yaml) due to "no space left on device". This happens during the container creation.
## Observations
Unlike other tasks of the role "get dbus-1 file" uses the default user and not root. The default user has very limited disk space (/dev/mapper/vg-lv_home 1.2G 630M 555M 54% /home) and watching disk space during deployment "/dev/mapper/vg-lv_home" spikes up close to 100% usage which might be the reason for the error.
## Possible Solution
Using "become: true" and therefore the root user in the "get dbus-1 file" tasks fixes the problem.
## Steps to reproduce
Add "/usr/share/openstack-tripleo-heat-templates/environments/manila-cephfsganesha-config.yaml" to the deployment.
## Error
(the registry url has been chagned due to security reasons)
{
"changed": false,
"cmd": [
"podman",
"run",
"--rm",
"--entrypoint=cat",
"os-registry.infra.h-da.cloud:5000/tripleowallabycentos9/daemon:current-ceph",
"/etc/dbus-1/system.d/org.ganesha.nfsd.conf"
],
"delta": "0:00:20.367417",
"end": "2023-06-16 10:56:14.870843",
"msg": "non-zero return code",
"rc": 125,
"start": "2023-06-16 10:55:54.503426",
"stderr": "Trying to pull registry:5000/tripleowallabycentos9/daemon:current-ceph...\nGetting image source signatures\nCopying blob sha256:bb9e6989315f6f108f7cab8ae301e09c9e41a982d776c25860169efaeb5f7530\nCopying blob sha256:f1ee40d9db4a2bf9b96ea48d6cb45c602a6761650f67dc84bba5a0d2495e845a\nCopying blob sha256:0d557d32f54ebd277fdffbbdf656b90442ee9d8753aec9ebac429eee967f4dee\nCopying blob sha256:6c5de04c936da27e33992af1e54e929f1cb39c8e1473d9d25ed1f1dc2d842fd4\nCopying blob sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\nCopying blob sha256:17facd475902d6709cff908630b59271c7ad18f64c3a1d0143d438c6988504ef\nCopying blob sha256:fbd68c38accbcb670817752f4faa8f3f845774fab06dc0b5e4b8c860f787ecd3\nCopying blob sha256:a57fb466f17b81e5b74877994d78b14170898db92c980ec5842dcfcbeefb324f\nCopying blob sha256:24972a36f4e1f7d9063e3cbc4ff3c171371cdeee9529e5581ab6e352a574b818\nCopying blob sha256:f48748416b3be33005287f0c884e0f98eebd74037311be116f8eff3cd85c3d51\nCopying blob sha256:e6e5b41683139f3942ff86736d3d416020b9b38c7d118df4ccf71e6809136c65\nCopying blob sha256:93cc817d8e84cb7580ee7d1930702f1ec087a93bcb30534a0c23341344113bb4\nCopying blob sha256:6e10a1f887f8a529f097f805ce4f6f64b06f13cfb638a6e1322bebe5113a86f4\nError: writing blob: adding layer with blob \"sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\": processing tar file(write /usr/lib64/samba/libsmbd-base-samba4.so: no space lefton device): exit status 1",
"stderr_lines": [
"Trying to pull registry:5000/tripleowallabycentos9/daemon:current-ceph...",
"Getting image source signatures",
"Copying blob sha256:bb9e6989315f6f108f7cab8ae301e09c9e41a982d776c25860169efaeb5f7530",
"Copying blob sha256:f1ee40d9db4a2bf9b96ea48d6cb45c602a6761650f67dc84bba5a0d2495e845a",
"Copying blob sha256:0d557d32f54ebd277fdffbbdf656b90442ee9d8753aec9ebac429eee967f4dee",
"Copying blob sha256:6c5de04c936da27e33992af1e54e929f1cb39c8e1473d9d25ed1f1dc2d842fd4",
"Copying blob sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5",
"Copying blob sha256:17facd475902d6709cff908630b59271c7ad18f64c3a1d0143d438c6988504ef",
"Copying blob sha256:fbd68c38accbcb670817752f4faa8f3f845774fab06dc0b5e4b8c860f787ecd3",
"Copying blob sha256:a57fb466f17b81e5b74877994d78b14170898db92c980ec5842dcfcbeefb324f",
"Copying blob sha256:24972a36f4e1f7d9063e3cbc4ff3c171371cdeee9529e5581ab6e352a574b818",
"Copying blob sha256:f48748416b3be33005287f0c884e0f98eebd74037311be116f8eff3cd85c3d51",
"Copying blob sha256:e6e5b41683139f3942ff86736d3d416020b9b38c7d118df4ccf71e6809136c65",
"Copying blob sha256:93cc817d8e84cb7580ee7d1930702f1ec087a93bcb30534a0c23341344113bb4",
"Copying blob sha256:6e10a1f887f8a529f097f805ce4f6f64b06f13cfb638a6e1322bebe5113a86f4",
"Error:writing blob: adding layer with blob \"sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\": processing tar file(write /usr/lib64/samba/libsmbd-base-samba4.so: no space left on device): exit status 1"
],
"stdout": "",
"stdout_lines": []
} |
|
| 2023-08-11 11:34:05 |
Lukas Koenen |
description |
While deploying manila using the "manila-cephfsganesha-config" heat template the deployment fails during the "get dbus-1 file" task which is part of the "tripleo_cephadm" ansible role (tasks/nfs.yaml) due to "no space left on device". This happens during the container creation.
## Observations
Unlike other tasks of the role "get dbus-1 file" uses the default user and not root. The default user has very limited disk space (/dev/mapper/vg-lv_home 1.2G 630M 555M 54% /home) and watching disk space during deployment "/dev/mapper/vg-lv_home" spikes up close to 100% usage which might be the reason for the error.
## Possible Solution
Using "become: true" and therefore the root user in the "get dbus-1 file" tasks fixes the problem.
## Steps to reproduce
Add "/usr/share/openstack-tripleo-heat-templates/environments/manila-cephfsganesha-config.yaml" to the deployment.
## Error
(the registry url has been chagned due to security reasons)
{
"changed": false,
"cmd": [
"podman",
"run",
"--rm",
"--entrypoint=cat",
"os-registry.infra.h-da.cloud:5000/tripleowallabycentos9/daemon:current-ceph",
"/etc/dbus-1/system.d/org.ganesha.nfsd.conf"
],
"delta": "0:00:20.367417",
"end": "2023-06-16 10:56:14.870843",
"msg": "non-zero return code",
"rc": 125,
"start": "2023-06-16 10:55:54.503426",
"stderr": "Trying to pull registry:5000/tripleowallabycentos9/daemon:current-ceph...\nGetting image source signatures\nCopying blob sha256:bb9e6989315f6f108f7cab8ae301e09c9e41a982d776c25860169efaeb5f7530\nCopying blob sha256:f1ee40d9db4a2bf9b96ea48d6cb45c602a6761650f67dc84bba5a0d2495e845a\nCopying blob sha256:0d557d32f54ebd277fdffbbdf656b90442ee9d8753aec9ebac429eee967f4dee\nCopying blob sha256:6c5de04c936da27e33992af1e54e929f1cb39c8e1473d9d25ed1f1dc2d842fd4\nCopying blob sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\nCopying blob sha256:17facd475902d6709cff908630b59271c7ad18f64c3a1d0143d438c6988504ef\nCopying blob sha256:fbd68c38accbcb670817752f4faa8f3f845774fab06dc0b5e4b8c860f787ecd3\nCopying blob sha256:a57fb466f17b81e5b74877994d78b14170898db92c980ec5842dcfcbeefb324f\nCopying blob sha256:24972a36f4e1f7d9063e3cbc4ff3c171371cdeee9529e5581ab6e352a574b818\nCopying blob sha256:f48748416b3be33005287f0c884e0f98eebd74037311be116f8eff3cd85c3d51\nCopying blob sha256:e6e5b41683139f3942ff86736d3d416020b9b38c7d118df4ccf71e6809136c65\nCopying blob sha256:93cc817d8e84cb7580ee7d1930702f1ec087a93bcb30534a0c23341344113bb4\nCopying blob sha256:6e10a1f887f8a529f097f805ce4f6f64b06f13cfb638a6e1322bebe5113a86f4\nError: writing blob: adding layer with blob \"sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\": processing tar file(write /usr/lib64/samba/libsmbd-base-samba4.so: no space lefton device): exit status 1",
"stderr_lines": [
"Trying to pull registry:5000/tripleowallabycentos9/daemon:current-ceph...",
"Getting image source signatures",
"Copying blob sha256:bb9e6989315f6f108f7cab8ae301e09c9e41a982d776c25860169efaeb5f7530",
"Copying blob sha256:f1ee40d9db4a2bf9b96ea48d6cb45c602a6761650f67dc84bba5a0d2495e845a",
"Copying blob sha256:0d557d32f54ebd277fdffbbdf656b90442ee9d8753aec9ebac429eee967f4dee",
"Copying blob sha256:6c5de04c936da27e33992af1e54e929f1cb39c8e1473d9d25ed1f1dc2d842fd4",
"Copying blob sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5",
"Copying blob sha256:17facd475902d6709cff908630b59271c7ad18f64c3a1d0143d438c6988504ef",
"Copying blob sha256:fbd68c38accbcb670817752f4faa8f3f845774fab06dc0b5e4b8c860f787ecd3",
"Copying blob sha256:a57fb466f17b81e5b74877994d78b14170898db92c980ec5842dcfcbeefb324f",
"Copying blob sha256:24972a36f4e1f7d9063e3cbc4ff3c171371cdeee9529e5581ab6e352a574b818",
"Copying blob sha256:f48748416b3be33005287f0c884e0f98eebd74037311be116f8eff3cd85c3d51",
"Copying blob sha256:e6e5b41683139f3942ff86736d3d416020b9b38c7d118df4ccf71e6809136c65",
"Copying blob sha256:93cc817d8e84cb7580ee7d1930702f1ec087a93bcb30534a0c23341344113bb4",
"Copying blob sha256:6e10a1f887f8a529f097f805ce4f6f64b06f13cfb638a6e1322bebe5113a86f4",
"Error:writing blob: adding layer with blob \"sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\": processing tar file(write /usr/lib64/samba/libsmbd-base-samba4.so: no space left on device): exit status 1"
],
"stdout": "",
"stdout_lines": []
} |
While deploying manila using the "manila-cephfsganesha-config" heat template the deployment fails during the "get dbus-1 file" task which is part of the "tripleo_cephadm" ansible role (tasks/nfs.yaml) due to "no space left on device". This happens during the container creation.
## Observations
Unlike other tasks of the role "get dbus-1 file" uses the default user and not root. The default user has very limited disk space (/dev/mapper/vg-lv_home 1.2G 630M 555M 54% /home) and watching disk space during deployment "/dev/mapper/vg-lv_home" spikes up close to 100% usage which might be the reason for the error.
## Possible Solution
Using "become: true" and therefore the root user in the "get dbus-1 file" tasks fixes the problem.
## Steps to reproduce
Add "/usr/share/openstack-tripleo-heat-templates/environments/manila-cephfsganesha-config.yaml" to the deployment.
## Error
(the registry url has been chagned due to security reasons)
{
"changed": false,
"cmd": [
"podman",
"run",
"--rm",
"--entrypoint=cat",
"registry:5000/tripleowallabycentos9/daemon:current-ceph",
"/etc/dbus-1/system.d/org.ganesha.nfsd.conf"
],
"delta": "0:00:20.367417",
"end": "2023-06-16 10:56:14.870843",
"msg": "non-zero return code",
"rc": 125,
"start": "2023-06-16 10:55:54.503426",
"stderr": "Trying to pull registry:5000/tripleowallabycentos9/daemon:current-ceph...\nGetting image source signatures\nCopying blob sha256:bb9e6989315f6f108f7cab8ae301e09c9e41a982d776c25860169efaeb5f7530\nCopying blob sha256:f1ee40d9db4a2bf9b96ea48d6cb45c602a6761650f67dc84bba5a0d2495e845a\nCopying blob sha256:0d557d32f54ebd277fdffbbdf656b90442ee9d8753aec9ebac429eee967f4dee\nCopying blob sha256:6c5de04c936da27e33992af1e54e929f1cb39c8e1473d9d25ed1f1dc2d842fd4\nCopying blob sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\nCopying blob sha256:17facd475902d6709cff908630b59271c7ad18f64c3a1d0143d438c6988504ef\nCopying blob sha256:fbd68c38accbcb670817752f4faa8f3f845774fab06dc0b5e4b8c860f787ecd3\nCopying blob sha256:a57fb466f17b81e5b74877994d78b14170898db92c980ec5842dcfcbeefb324f\nCopying blob sha256:24972a36f4e1f7d9063e3cbc4ff3c171371cdeee9529e5581ab6e352a574b818\nCopying blob sha256:f48748416b3be33005287f0c884e0f98eebd74037311be116f8eff3cd85c3d51\nCopying blob sha256:e6e5b41683139f3942ff86736d3d416020b9b38c7d118df4ccf71e6809136c65\nCopying blob sha256:93cc817d8e84cb7580ee7d1930702f1ec087a93bcb30534a0c23341344113bb4\nCopying blob sha256:6e10a1f887f8a529f097f805ce4f6f64b06f13cfb638a6e1322bebe5113a86f4\nError: writing blob: adding layer with blob \"sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\": processing tar file(write /usr/lib64/samba/libsmbd-base-samba4.so: no space lefton device): exit status 1",
"stderr_lines": [
"Trying to pull registry:5000/tripleowallabycentos9/daemon:current-ceph...",
"Getting image source signatures",
"Copying blob sha256:bb9e6989315f6f108f7cab8ae301e09c9e41a982d776c25860169efaeb5f7530",
"Copying blob sha256:f1ee40d9db4a2bf9b96ea48d6cb45c602a6761650f67dc84bba5a0d2495e845a",
"Copying blob sha256:0d557d32f54ebd277fdffbbdf656b90442ee9d8753aec9ebac429eee967f4dee",
"Copying blob sha256:6c5de04c936da27e33992af1e54e929f1cb39c8e1473d9d25ed1f1dc2d842fd4",
"Copying blob sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5",
"Copying blob sha256:17facd475902d6709cff908630b59271c7ad18f64c3a1d0143d438c6988504ef",
"Copying blob sha256:fbd68c38accbcb670817752f4faa8f3f845774fab06dc0b5e4b8c860f787ecd3",
"Copying blob sha256:a57fb466f17b81e5b74877994d78b14170898db92c980ec5842dcfcbeefb324f",
"Copying blob sha256:24972a36f4e1f7d9063e3cbc4ff3c171371cdeee9529e5581ab6e352a574b818",
"Copying blob sha256:f48748416b3be33005287f0c884e0f98eebd74037311be116f8eff3cd85c3d51",
"Copying blob sha256:e6e5b41683139f3942ff86736d3d416020b9b38c7d118df4ccf71e6809136c65",
"Copying blob sha256:93cc817d8e84cb7580ee7d1930702f1ec087a93bcb30534a0c23341344113bb4",
"Copying blob sha256:6e10a1f887f8a529f097f805ce4f6f64b06f13cfb638a6e1322bebe5113a86f4",
"Error:writing blob: adding layer with blob \"sha256:49a61133ba378a1ca5e8f77b3c05ed9980f80e89ff729db22a51bb524363e6a5\": processing tar file(write /usr/lib64/samba/libsmbd-base-samba4.so: no space left on device): exit status 1"
],
"stdout": "",
"stdout_lines": []
} |
|
| 2023-08-11 11:37:11 |
Lukas Koenen |
bug task added |
|
tripleo |
|
| 2023-08-15 17:08:14 |
Vida Haririan |
tags |
|
ganesha |
|
| 2023-08-17 07:29:57 |
Manojkatari |
manila: assignee |
|
Manojkatari (mkatari) |
|
| 2023-08-17 15:47:22 |
Vida Haririan |
manila: status |
New |
Invalid |
|
| 2023-08-22 05:30:15 |
Takashi Kajinami |
tripleo: assignee |
|
Manojkatari (mkatari) |
|
| 2023-08-22 05:30:18 |
Takashi Kajinami |
manila: assignee |
Manojkatari (mkatari) |
|
|
