luminous: Couldn't init storage provider (RADOS)

Bug #1775623 reported by Josh Kirkpatrick
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ceph RADOS Gateway Charm
Invalid
Undecided
Unassigned
OpenStack Glance Charm
Invalid
Undecided
Unassigned

Bug Description

I am unable to initialize the radosgw service in the ceph-radosgw charm. For versions radosgw/257 and /256. I searched and appears to be an issue with rados. See bug report >> http://tracker.ceph.com/issues/23349.

It states the but is planned to be fixed for 12.2.5, however current releaased charms has only up to 12.2.4

  ceph-radosgw:
    charm: cs:ceph-radosgw-256
    num_units: 1
    options:
      os-admin-network: 172.x.x.0/24
      os-internal-network: 172.x.x.0/24
      os-public-network: 10.y.y.y/27
      source: 'cloud:xenial-queens'
    constraints: 'spaces=default,os-internal-net'
    expose: true
    annotations:
      gui-x: '1000'
      gui-y: '250'
    to:
      - '0'
Relations:
    - - 'ceph-radosgw:mon'
    - 'ceph-mon:radosgw'
  - - 'ceph-radosgw:identity-service'

root@juju-fe4a6f-0-lxd-2:/var/log/apache2# ceph --version
ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) luminous (stable)

Please note I did perform the following :
    juju ssh ceph/0 \
      'sudo radosgw-admin user create --uid="ubuntu" --display-name="Ubuntu Ceph"'

user@us-a-reg-maas:~/openstack-base$ juju ssh ceph-mon/0
ubuntu@juju-fe4a6f-1-lxd-0:~$ sudo radosgw-admin user list
[
    "ubuntu"
]

I am attaching logs that I believe are useful and necessary.

Revision history for this message
Josh Kirkpatrick (rockfreak101) wrote :
Revision history for this message
Josh Kirkpatrick (rockfreak101) wrote :

Added glance to impacted projects, because if ceph-radosgw is deployed with glance, while glance uses ceph, it will appear that glance times out, but in fact its that the radosgw is unreachable.

Revision history for this message
Josh Kirkpatrick (rockfreak101) wrote :

I found the issue myself. the glance charm requires either additional routing provided, or has the storage networks on the charm as well. Once adding the storage networks to the glance charm, I was able to connect to radosgw.

Revision history for this message
Dmitrii Shcherbakov (dmitriis) wrote :

It's a subtle requirement but you are right that glance <-> ceph connectivity is required as glance stores images in Ceph and, therefore, needs access to both monitors and OSDs.

In general, any ceph client needs the same type of network connectivity whether it's QEMU (nova), glance, cinder, rados-gateway, gnocchi or something else.

You could place a constraint or add a binding to glance (ceph: *ceph-access-space):

variables:
  ceph-access-constr: &ceph-access-constr spaces=ceph-access-space

#...
  glance:
    charm: cs:xenial/glance
    constraints: *ceph-access-constr # <-
    bindings:
      "": *oam-space
      public: *public-space
      admin: *admin-space
      internal: *internal-space
      shared-db: *internal-space

Revision history for this message
Dmitrii Shcherbakov (dmitriis) wrote :

Josh,

I'll mark this bug as invalid if you don't mind.

Changed in charm-ceph-radosgw:
status: New → Invalid
Changed in charm-glance:
status: New → Invalid
Revision history for this message
Josh Kirkpatrick (rockfreak101) wrote : Re: [Bug 1775623] Re: luminous: Couldn't init storage provider (RADOS)

I guess that's fine. Maybe there is an opportunity for me though
documentation in one of the charms?

But correct, this is not a bug, you may close.

On Mon, Jun 18, 2018, 12:20 Dmitrii Shcherbakov <email address hidden>
wrote:

> Josh,
>
> I'll mark this bug as invalid if you don't mind.
>
> ** Changed in: charm-ceph-radosgw
> Status: New => Invalid
>
> ** Changed in: charm-glance
> Status: New => Invalid
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1775623
>
> Title:
> luminous: Couldn't init storage provider (RADOS)
>
> Status in OpenStack ceph-radosgw charm:
> Invalid
> Status in OpenStack glance charm:
> Invalid
>
> Bug description:
> I am unable to initialize the radosgw service in the ceph-radosgw
> charm. For versions radosgw/257 and /256. I searched and appears to be
> an issue with rados. See bug report >>
> http://tracker.ceph.com/issues/23349.
>
> It states the but is planned to be fixed for 12.2.5, however current
> releaased charms has only up to 12.2.4
>
> ceph-radosgw:
> charm: cs:ceph-radosgw-256
> num_units: 1
> options:
> os-admin-network: 172.x.x.0/24
> os-internal-network: 172.x.x.0/24
> os-public-network: 10.y.y.y/27
> source: 'cloud:xenial-queens'
> constraints: 'spaces=default,os-internal-net'
> expose: true
> annotations:
> gui-x: '1000'
> gui-y: '250'
> to:
> - '0'
> Relations:
> - - 'ceph-radosgw:mon'
> - 'ceph-mon:radosgw'
> - - 'ceph-radosgw:identity-service'
>
>
> root@juju-fe4a6f-0-lxd-2:/var/log/apache2# ceph --version
> ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) luminous
> (stable)
>
>
> Please note I did perform the following :
> juju ssh ceph/0 \
> 'sudo radosgw-admin user create --uid="ubuntu"
> --display-name="Ubuntu Ceph"'
>
> user@us-a-reg-maas:~/openstack-base$ juju ssh ceph-mon/0
> ubuntu@juju-fe4a6f-1-lxd-0:~$ sudo radosgw-admin user list
> [
> "ubuntu"
> ]
>
>
> I am attaching logs that I believe are useful and necessary.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/charm-ceph-radosgw/+bug/1775623/+subscriptions
>

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to charm-glance (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/576230

Revision history for this message
Dmitrii Shcherbakov (dmitriis) wrote :

Josh,

I think you're right, it is quite frustrating to debug.

I had to do this at least once myself - raised a review to document the requirement.

Revision history for this message
Josh Kirkpatrick (rockfreak101) wrote : Re: [Bug 1775623] Re: luminous: Couldn't init storage provider (RADOS)

Fantastic

On Mon, Jun 18, 2018, 13:50 Dmitrii Shcherbakov <email address hidden>
wrote:

> Josh,
>
> I think you're right, it is quite frustrating to debug.
>
> I had to do this at least once myself - raised a review to document the
> requirement.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1775623
>
> Title:
> luminous: Couldn't init storage provider (RADOS)
>
> Status in OpenStack ceph-radosgw charm:
> Invalid
> Status in OpenStack glance charm:
> Invalid
>
> Bug description:
> I am unable to initialize the radosgw service in the ceph-radosgw
> charm. For versions radosgw/257 and /256. I searched and appears to be
> an issue with rados. See bug report >>
> http://tracker.ceph.com/issues/23349.
>
> It states the but is planned to be fixed for 12.2.5, however current
> releaased charms has only up to 12.2.4
>
> ceph-radosgw:
> charm: cs:ceph-radosgw-256
> num_units: 1
> options:
> os-admin-network: 172.x.x.0/24
> os-internal-network: 172.x.x.0/24
> os-public-network: 10.y.y.y/27
> source: 'cloud:xenial-queens'
> constraints: 'spaces=default,os-internal-net'
> expose: true
> annotations:
> gui-x: '1000'
> gui-y: '250'
> to:
> - '0'
> Relations:
> - - 'ceph-radosgw:mon'
> - 'ceph-mon:radosgw'
> - - 'ceph-radosgw:identity-service'
>
>
> root@juju-fe4a6f-0-lxd-2:/var/log/apache2# ceph --version
> ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) luminous
> (stable)
>
>
> Please note I did perform the following :
> juju ssh ceph/0 \
> 'sudo radosgw-admin user create --uid="ubuntu"
> --display-name="Ubuntu Ceph"'
>
> user@us-a-reg-maas:~/openstack-base$ juju ssh ceph-mon/0
> ubuntu@juju-fe4a6f-1-lxd-0:~$ sudo radosgw-admin user list
> [
> "ubuntu"
> ]
>
>
> I am attaching logs that I believe are useful and necessary.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/charm-ceph-radosgw/+bug/1775623/+subscriptions
>

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to charm-glance (master)

Reviewed: https://review.openstack.org/576230
Committed: https://git.openstack.org/cgit/openstack/charm-glance/commit/?id=64180d831d68d12f301fb374da22a1fea9c87058
Submitter: Zuul
Branch: master

commit 64180d831d68d12f301fb374da22a1fea9c87058
Author: Dmitrii Shcherbakov <email address hidden>
Date: Mon Jun 18 21:38:34 2018 +0300

    add ceph ip connectivity requirement clarification

    Change-Id: I0f6159309260f6af3968729bb57b382095aec813
    Related-Bug: #1775623

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.