Zun

The Kolla's zun_compute container failed

Bug #1795262 reported by hongbin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Zun
Fix Released
High
hongbin

Bug Description

Use kolla-ansible to deploy Zun. The zun_compute container failed with the following error:

2018-09-30 02:40:31.082 29 DEBUG oslo.privsep.daemon [-] privsep: reply[139973143560272]: (5, 'zun.common.exception.CommandError', (u"The command: df -B 1G /var/lib/docker failed on the system, due to Unexpected error while running command.\nCommand: df -B 1G /var/lib/docker\nExit code: 1\nStdout: u''\nStderr: u'df: /var/lib/docker: No such file or directory\\n'",)) loop /var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_privsep/daemon.py:456
2018-09-30 02:40:31.085 5 INFO zun.container.docker.driver [-] There was a problem while executing df -B 1G /var/lib/docker
2018-09-30 02:40:31.085 5 ERROR zun.common.exception [-] Exception in string format operation, kwargs: {'code': 500, 'cmd': 'df'}: KeyError: 'error'
2018-09-30 02:40:31.085 5 ERROR zun.common.exception Traceback (most recent call last):
2018-09-30 02:40:31.085 5 ERROR zun.common.exception File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/zun/common/exception.py", line 204, in __init__
2018-09-30 02:40:31.085 5 ERROR zun.common.exception self.message = str(self.message) % kwargs
2018-09-30 02:40:31.085 5 ERROR zun.common.exception KeyError: 'error'
2018-09-30 02:40:31.085 5 ERROR zun.common.exception
2018-09-30 02:40:31.091 5 ERROR oslo_service.periodic_task [-] Error during Manager.inventory_host: CommandError: The command: %(cmd)s failed on the system, due to %(error)s
2018-09-30 02:40:31.091 5 ERROR oslo_service.periodic_task Traceback (most recent call last):
2018-09-30 02:40:31.091 5 ERROR oslo_service.periodic_task File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
2018-09-30 02:40:31.091 5 ERROR oslo_service.periodic_task task(self, context)
2018-09-30 02:40:31.091 5 ERROR oslo_service.periodic_task File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/zun/compute/manager.py", line 1078, in inventory_host
2018-09-30 02:40:31.091 5 ERROR oslo_service.periodic_task rt.update_available_resources(context)
2018-09-30 02:40:31.091 5 ERROR oslo_service.periodic_task File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/zun/compute/compute_node_tracker.py", line 65, in update_available_resources
2018-09-30 02:40:31.091 5 ERROR oslo_service.periodic_task self.container_driver.get_available_resources(node)
2018-09-30 02:40:31.091 5 ERROR oslo_service.periodic_task File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/zun/container/driver.py", line 251, in get_available_resources
2018-09-30 02:40:31.091 5 ERROR oslo_service.periodic_task disk_total = self.get_total_disk_for_container()
2018-09-30 02:40:31.091 5 ERROR oslo_service.periodic_task File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/zun/container/docker/driver.py", line 1127, in get_total_disk_for_container
2018-09-30 02:40:31.091 5 ERROR oslo_service.periodic_task raise exception.CommandError(cmd='df')
2018-09-30 02:40:31.091 5 ERROR oslo_service.periodic_task CommandError: The command: %(cmd)s failed on the system, due to %(error)s
2018-09-30 02:40:31.091 5 ERROR oslo_service.periodic_task

hongbin (hongbin034)
Changed in zun:
importance: Undecided → High
status: New → Triaged
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to zun (master)

Fix proposed to branch: master
Review: https://review.openstack.org/607459

Changed in zun:
assignee: nobody → hongbin (hongbin034)
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to zun (master)

Reviewed: https://review.openstack.org/607459
Committed: https://git.openstack.org/cgit/openstack/zun/commit/?id=d71af9ea4416b1292a0b22388ca2fff58538e036
Submitter: Zuul
Branch: master

commit d71af9ea4416b1292a0b22388ca2fff58538e036
Author: Hongbin Lu <email address hidden>
Date: Wed Oct 3 03:55:53 2018 +0000

    Handle docker data root doesn't exist

    In kolla, zun compute run inside a container so it couldn't
    access the docker data root. Zun needs to handle this case.

    Change-Id: I6882803404a1a5a6a5fe59afb1ee4b63bcd36514
    Closes-Bug: #1795262

Changed in zun:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/zun 3.0.0.0rc1

This issue was fixed in the openstack/zun 3.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.