glance-api-service fails to start trying to find 'sheepdog' when there's no reference of it in glance-api.conf

[Kashyap Chamarthy]

Version Info
-------------

http://repos.fedorapeople.org/repos/openstack/openstack-trunk/fedora-openstack-trunk.repo

    $ rpm -qa | egrep -i 'glance'
    python-glanceclient-0.9.0.49.g29270bb-0.1.295.29270bb_9567343.noarch
    python-glance-2013.2.a116.gd13493b-0.1.2712.d13493b_c29d712.noarch
    openstack-glance-2013.2.a116.gd13493b-0.1.2712.d13493b_c29d712.noarch

Description
--------------

I'm just trying to configure OpenStack services, manually, in a large KVM based guest (Fedora 19). These are Havana trunk bits. By default Sheepdog section was enabled, I commented out those services (refer below for config files).

Instructions to reproduce
------------------------------

1/ Configure Glance service manually (Assumes Keystone is configured successfully):

    $ yum install openstack-glance -y
    $ openstack-db --init --service glance
    $ keystone tenant-create --name services
    $ keystone user-create --name glance --pass foobar
    $ keystone user-role-add --user glance \
       --role admin --tenant services

2/ Update the glance-api.conf::

    $ openstack-config --set /etc/glance/glance-api.conf \
      paste_deploy flavor keystone
    $ openstack-config --set /etc/glance/glance-api.conf \
      keystone_authtoken admin_tenant_name services
    $ openstack-config --set /etc/glance/glance-api.conf \
      keystone_authtoken admin_user glance
    $ openstack-config --set /etc/glance/glance-api.conf \
      keystone_authtoken admin_password foobar

3/ Update the glance-registry.conf::

    $ openstack-config --set /etc/glance/glance-registry.conf \
      paste_deploy flavor keystone
    $ openstack-config --set /etc/glance/glance-registry.conf \
      keystone_authtoken admin_tenant_name services
    $ openstack-config --set /etc/glance/glance-registry.conf \
      keystone_authtoken admin_user glance
    $ openstack-config --set /etc/glance/glance-registry.conf \
      keystone_authtoken admin_password foobar

4/ Start the glance-api.service

    $ systemctl start openstack-glance-api.service

5/ Check the status of glance-api.service:

    $ systemctl status openstack-glance-api.service
    openstack-glance-api.service - OpenStack Image Service (code-named Glance) API server
       Loaded: loaded (/usr/lib/systemd/system/openstack-glance-api.service; disabled)
       Active: failed (Result: exit-code) since Wed 2013-07-17 23:17:59 EDT; 634ms ago
      Process: 2230 ExecStart=/usr/bin/glance-api --config-file /etc/glance/glance-api.conf (code=exited, status=1/FAILURE)

    Jul 17 23:17:59 fedostk glance-api[2230]: Stdout: ''
    Jul 17 23:17:59 fedostk glance-api[2230]: Stderr: '/bin/sh: collie: command not found\n'
    Jul 17 23:17:59 fedostk glance-api[2230]: 2013-07-17 23:17:59.907 2230 WARNING glance.store.base [-] Failed to configure store correctly: Store sheepdog could not be configured correctly. Reason...nning command.
    Jul 17 23:17:59 fedostk glance-api[2230]: Command: collie
    Jul 17 23:17:59 fedostk glance-api[2230]: Exit code: 127
    Jul 17 23:17:59 fedostk glance-api[2230]: Stdout: ''
    Jul 17 23:17:59 fedostk glance-api[2230]: Stderr: '/bin/sh: collie: command not found\n' Disabling add method.
    Jul 17 23:17:59 fedostk glance-api[2230]: 2013-07-17 23:17:59.909 2230 CRITICAL glance [-] No module named cinderclient
    Jul 17 23:17:59 fedostk systemd[1]: openstack-glance-api.service: main process exited, code=exited, status=1/FAILURE
    Jul 17 23:17:59 fedostk systemd[1]: Unit openstack-glance-api.service entered failed state.

Log information
------------------

-> Glance api.log:
    $ tail /var/log/glance/api.log
    Command: collie
    Exit code: 127
    Stdout: ''
    Stderr: '/bin/sh: collie: command not found\n'
    2013-07-17 22:33:38.021 2138 WARNING glance.store.base [-] Failed to configure store correctly: Store sheepdog could not be configured correctly. Reason: Error in store configuration: Unexpected error while running command.
    Command: collie
    Exit code: 127
    Stdout: ''
    Stderr: '/bin/sh: collie: command not found\n' Disabling add method.
    2013-07-17 22:33:38.023 2138 CRITICAL glance [-] No module named cinderclient
:nohls

Configuration files
---------------------

-> glance-cache.conf

    $ cat /etc/glance/glance-cache.conf | grep -v ^$ | grep -v ^#
    [DEFAULT]
    log_file = /var/log/glance/image-cache.log
    image_cache_dir = /var/lib/glance/image-cache/
    image_cache_stall_time = 86400
    image_cache_invalid_entry_grace_period = 3600
    image_cache_max_size = 10737418240
    registry_host = 0.0.0.0
    registry_port = 9191
    admin_tenant_name = services
    admin_user = glance
    filesystem_store_datadir = /var/lib/glance/images/

-> glance-api.conf

    $ cat /etc/glance/glance-api.conf | grep -v ^$ | grep -v ^#
    [DEFAULT]
    default_store = file
    bind_host = 0.0.0.0
    bind_port = 9292
    log_file = /var/log/glance/api.log
    backlog = 4096
    sql_connection = mysql://glance:glance@localhost/glance
    sql_idle_timeout = 3600
    workers = 1
    filesystem_store_datadir = /var/lib/glance/images/
    swift_store_auth_version = 2
    swift_store_auth_address = 127.0.0.1:5000/v2.0/
    swift_store_user = jdoe:jdoe
    swift_store_key = a86850deb2742ec3cb41518e26aa2d89
    swift_store_container = glance
    swift_store_create_container_on_put = False
    swift_store_large_object_size = 5120
    swift_store_large_object_chunk_size = 200
    swift_enable_snet = False
    delayed_delete = False
    scrub_time = 43200
    scrubber_datadir = /var/lib/glance/scrubber
    image_cache_dir = /var/lib/glance/image-cache/
    [keystone_authtoken]
    auth_host = 127.0.0.1
    auth_port = 35357
    auth_protocol = http
    admin_tenant_name = services
    admin_user = glance
    admin_password = foobar
    [paste_deploy]
    flavor = keystone

Other information
---------------------

glance-registry service runs just fine:

    $ systemctl status openstack-glance-registry.service
    openstack-glance-registry.service - OpenStack Image Service (code-named Glance) Registry server
       Loaded: loaded (/usr/lib/systemd/system/openstack-glance-registry.service; enabled)
       Active: active (running) since Wed 2013-07-17 23:17:45 EDT; 4s ago
     Main PID: 2220 (glance-registry)
       CGroup: name=systemd:/system/openstack-glance-registry.service
               ├─2220 /usr/bin/python /usr/bin/glance-registry --config-file /etc/glance/glance-registry.conf
               └─2225 /usr/bin/python /usr/bin/glance-registry --config-file /etc/glance/glance-registry.conf

    Jul 17 23:17:45 fedostk systemd[1]: Started OpenStack Image Service (code-named Glance) Registry server

[Kashyap Chamarthy (kashyapc)]

Hm, on further investigation:

Just to try further, I installed Sheepdog and cinderclient (complianed above), and tried to start glance-api.service:

    $ yum install sheepdog python-cinderclient -y
    $ systemctl start glance-api.service

The service starts successfully:

    $ systemctl status openstack-glance-api.service
    openstack-glance-api.service - OpenStack Image Service (code-named Glance) API server
       Loaded: loaded (/usr/lib/systemd/system/openstack-glance-api.service; disabled)
       Active: active (running) since Wed 2013-07-17 23:30:13 EDT; 7s ago
     Main PID: 2286 (glance-api)
       CGroup: name=systemd:/system/openstack-glance-api.service
               ├─2286 /usr/bin/python /usr/bin/glance-api --config-file /etc/glance/glance-api.conf
               └─2292 /usr/bin/python /usr/bin/glance-api --config-file /etc/glance/glance-api.conf

    Jul 17 23:30:13 fedostk systemd[1]: Starting OpenStack Image Service (code-named Glance) API server...
    Jul 17 23:30:13 fedostk systemd[1]: Started OpenStack Image Service (code-named Glance) API server.
    Jul 17 23:30:14 fedostk glance-api[2286]: 2013-07-17 23:30:14.094 2286 WARNING glance.store.base [-] Failed to configure store correctly: Store s3 could not be configured correctly. Reason: Coul...ng add method.
    Jul 17 23:30:14 fedostk glance-api[2286]: 2013-07-17 23:30:14.171 2286 WARNING glance.store.base [-] Failed to configure store correctly: Store cinder could not be configured correctly. Reason: ...ng add method.

[Kashyap Chamarthy (kashyapc)]

Duh, it looks like glance-api service failed to start due to not having the cinderclient package (the CRITICAL error mentioned in api.log). I confirmed by removing the sheepdog package, and restarting the glance-api service.

If someone yells this is all kosher, I'll can close this bug. Just thought I'll track it here.

[Vivekanandan B (bvivek)]

It worked for me too... Thanks...

[Mitesh Shah (miteshshah)]

Glance api.log:

$ tail /var/log/glance/api.log
Command: collie
Exit code: 127
Stdout: ''
Stderr: '/bin/sh: 1: collie: not found\n' Disabling add method.
2013-10-28 10:12:46.716 6354 WARNING glance.store.base [-] Failed to configure store correctly: Store cinder could not be configured correctly. Reason: Cinder storage requires a context. Disabling add method.
2013-10-28 10:12:46.733 6354 WARNING glance.api.v2.images [-] Could not find schema properties file schema-image.json. Continuing without custom properties

[Mitesh Shah (miteshshah)]

On Ubuntu 12.04.3 x86_64

# apt-get install sheepdog python-cinderclient
# service glance-api restart