CentOS radosgw is not started after reboot

[Stanislav Makar]

First of all
[root@node-7 ~]# chkconfig --list | grep ceph
ceph 0:off 1:off 2:on 3:on 4:on 5:on 6:off

on conroller before reboot
[root@node-7 ~]# ps ax | grep radosg[w]
23876 ? Ssl 0:00 /usr/bin/radosgw -n client.radosgw.gateway
[root@node-7 ~]#

after reboot
[root@node-7 ~]# ps ax | grep radosg[w]
[root@node-7 ~]#

Below helps
[root@node-7 ~]# chkconfig ceph-radosgw on
[root@node-7 ~]# chkconfig --list | grep ceph
ceph 0:off 1:off 2:on 3:on 4:on 5:on 6:off
ceph-radosgw 0:off 1:off 2:on 3:on 4:on 5:on 6:off

cat /etc/fuel/version.yaml
VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "6.1"
  openstack_version: "2014.2.2-6.1"
  api: "1.0"
  build_number: "522"
  build_id: "2015-06-16_13-53-26"
  nailgun_sha: "fa8dec50f3df2626c97f6c38a897cf4e0f80b39d"
  python-fuelclient_sha: "4fc55db0265bbf39c369df398b9dc7d6469ba13b"
  astute_sha: "1ea8017fe8889413706d543a5b9f557f5414beae"
  fuel-library_sha: "3528dddbd0c961290909d5e3e256f55ff75cd2fc"
  fuel-ostf_sha: "8fefcf7c4649370f00847cc309c24f0b62de718d"
  fuelmain_sha: "42020c36d6dec9fedf61faa68aa3674156d41977"

+
yum install git && git clone https://github.com/stackforge/fuel-library.git library -b stable/6.1
cd library/deployment/puppet/ && rsync -av --delete * /etc/puppet/modules
fuel rel --sync-deployment-tasks --dir /etc/puppet/

git log --pretty=oneline
2e7a08ad9792c700ebf08ce87f4867df36aa9fab Revert "Fix the problem with ceph deployment on scale lab"
1698c27d807230b9039c547b9c1593ccbe189135 Merge "Remove raise Timeout::Error from ntp_available" into stable/6.1
14ac1f7025e7490975f8156f351f35d42b1f3672 Remove raise Timeout::Error from ntp_available
26b80a4515d53adb99458bd10d3018511f4a2a46 Make haproxy stats listen on lo and mgmt VIP
3528dddbd0c961290909d5e3e256f55ff75cd2fc Fix for stale arp table usage

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/6.1
Review: https://review.openstack.org/193246

[Alex Schultz (alex-schultz)]

master: https://review.openstack.org/#/c/193242/

stable/6.1: https://review.openstack.org/#/c/193246/

[Bogdan Dobrelya (bogdando)]

High bugs cannot be targeted for 6.1 because of HCF. Moved to 6.1-updates

[OpenStack Infra (hudson-openstack)]

Change abandoned by Alex Schultz (<email address hidden>) on branch: stable/6.1
Review: https://review.openstack.org/193246
Reason: abandoning until it gets sorted out in master

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/6.0
Review: https://review.openstack.org/194802

[Maksym Strukov (unbelll)]

Verified as fixed in 7.0-17

{"build_id": "2015-06-29_18-10-01", "build_number": "17", "release_versions": {"2014.2.2-7.0": {"VERSION": {"build_id": "2015-06-29_18-10-01", "build_number": "17", "api": "1.0", "fuel-library_sha": "29935efd3d6c0bfb385d14b94528df418ac81ef3", "nailgun_sha": "026d4ee9570fc844c9b312e3d52c643528b655f6", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-7.0", "production": "docker", "python-fuelclient_sha": "79f6129e3b6b440c96ac9e96041fad4fd2e64379", "astute_sha": "776157f722b13aff5f59bc098cf948793e6498ef", "fuel-ostf_sha": "69e7fa120e8efa7ed74d98efc63079d2f5c69d7b", "release": "7.0", "fuelmain_sha": "4f2dff3bdc327858fa45bcc2853cfbceae68a40c"}}}, "auth_required": true, "api": "1.0", "fuel-library_sha": "29935efd3d6c0bfb385d14b94528df418ac81ef3", "nailgun_sha": "026d4ee9570fc844c9b312e3d52c643528b655f6", "feature_groups": ["mirantis"], "openstack_version": "2014.2.2-7.0", "production": "docker", "python-fuelclient_sha": "79f6129e3b6b440c96ac9e96041fad4fd2e64379", "astute_sha": "776157f722b13aff5f59bc098cf948793e6498ef", "fuel-ostf_sha": "69e7fa120e8efa7ed74d98efc63079d2f5c69d7b", "release": "7.0", "fuelmain_sha": "4f2dff3bdc327858fa45bcc2853cfbceae68a40c"}

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/193246
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=89e793ef4a9e0b52f5d3f50b3f2c057be1385425
Submitter: Jenkins
Branch: stable/6.1

commit 89e793ef4a9e0b52f5d3f50b3f2c057be1385425
Author: Alex Schultz <email address hidden>
Date: Thu Jun 18 13:24:53 2015 -0500

    Ensure radosgw service enabled on boot

    Currently, we are not ensuring that the radowgw service is enabled to
    start on boot. This change removes the restart_rados.sh and adds
    enable_rados.pp to ensure the service is running and will be started
    on boot.

    Change-Id: I58bf4a20c9d08ba3e9624fd47e3b5780011c89b7
    Closes-Bug: 1466552
    (cherry picked from commit 23243f3c9f865bd329ed1e65ea5198f46e46b2ac)

[Vadim Rovachev (vrovachev)]

Verified patch for 6.1. Patch works.

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Alexey Shtokolov <email address hidden> on branch: stable/6.1
Review: https://review.fuel-infra.org/9101

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: stable/6.1
Change author: Alex Ermolov <email address hidden>
Review: https://review.fuel-infra.org/9164

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/9164
Submitter: Alexey Shtokolov <email address hidden>
Branch: stable/6.1

Commit: 9931f6e8cb8f755a9ef92428a1a9b3b979787fca
Author: Alex Ermolov <email address hidden>
Date: Wed Jul 8 18:02:06 2015

CentOS radosgw is not started after reboot

Change-Id: I8522524ad320aa80c38911546ce8bf72436e46ec
Closes-Bug: #1466552

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/6.0
Review: https://review.openstack.org/200070

[OpenStack Infra (hudson-openstack)]

Change abandoned by Alex Schultz (<email address hidden>) on branch: stable/6.0
Review: https://review.openstack.org/194802
Reason: https://review.openstack.org/#/c/200070/ has the correct change-id with this change

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: stable/6.1
Change author: Alex Ermolov <email address hidden>
Review: https://review.fuel-infra.org/9188

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/9188
Submitter: Vitaly Sedelnik <email address hidden>
Branch: stable/6.1

Commit: e09e7c646b7a019cf7269e16e382ae7e37a71c44
Author: Alex Ermolov <email address hidden>
Date: Fri Jul 10 12:09:56 2015

CentOS radosgw is not started after reboot

Change-Id: I115de981b8a2939432591ca9e87a908e2a1e7066
Closes-Bug: #1466552

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: stable/6.1
Change author: Alexey Shtokolov <email address hidden>
Review: https://review.fuel-infra.org/9251

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/9251
Submitter: Vitaly Sedelnik <email address hidden>
Branch: stable/6.1

Commit: 66b7212f5c4da11d2bb879b9a9891b2aed4ceaa3
Author: Alexey Shtokolov <email address hidden>
Date: Mon Jul 13 10:34:22 2015

Closes-Bug: #1466552

Change-Id: Ibc20150667e44c3bb79090dc04d1a2cb19b2319c

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: stable/6.1
Change author: Alexey Shtokolov <email address hidden>
Review: https://review.fuel-infra.org/9267

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/9267
Submitter: Alexey Shtokolov <email address hidden>
Branch: stable/6.1

Commit: 28b5d565ac4196c6179765b3fac977f33eacd6e1
Author: Alexey Shtokolov <email address hidden>
Date: Mon Jul 13 13:57:10 2015

Closes-Bug: #1466552

Change-Id: I47a4795f735ba5beb2cae55565446fa8cd333b89

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: stable/6.1
Change author: Alex Ermolov <email address hidden>
Review: https://review.fuel-infra.org/9353

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Alex Ermolov <email address hidden> on branch: stable/6.1
Review: https://review.fuel-infra.org/9353

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: stable/6.1
Change author: Alex Ermolov <email address hidden>
Review: https://review.fuel-infra.org/9355

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/9355
Submitter: Vitaly Sedelnik <email address hidden>
Branch: stable/6.1

Commit: fc1ff7645ad277d4230bc7fd2b3bdfe7a80465fc
Author: Alex Ermolov <email address hidden>
Date: Wed Jul 15 14:00:17 2015

CentOS radosgw doesn't start after reboot

Change-Id: I4ccbd3417b391f193cb31db20f134d224d90b694
Closes-Bug: #1466552

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: stable/6.1
Change author: Vitaly Sedelnik <email address hidden>
Review: https://review.fuel-infra.org/9627

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/9627
Submitter: Vitaly Sedelnik <email address hidden>
Branch: stable/6.1

Commit: 9392b34251b6d9de73225439729e51c7b2d65055
Author: Vitaly Sedelnik <email address hidden>
Date: Mon Jul 20 13:17:18 2015

Sync deployment tasks for radosgw issue

Fix for radosgw issue changes deployment tasks so they should
be synchronized after updating the packages. This change adds
sync-deployment-tasks invocation to patching scenario after
yum update.

Related-Bug: #1466552

Change-Id: Ia943cd56489197ef6d1da4e60cd1be61b4926134

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/200070
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=cc91692530a702c697a555111089f890dcc43fba
Submitter: Jenkins
Branch: stable/6.0

commit cc91692530a702c697a555111089f890dcc43fba
Author: Alex Schultz <email address hidden>
Date: Tue Jun 23 15:21:35 2015 -0500

    Ensure radosgw service enabled on boot

    Currently, we are not ensuring that the radowgw service is enabled to
    start on boot. This change adds the radosgw service back in but only to
    use the enable property.

    Change-Id: I58bf4a20c9d08ba3e9624fd47e3b5780011c89b7
    Closes-Bug: 1466552

[OSCI Robot (oscirobot)]

Changeset merged. Package placed on primary repository.
RPM package fuel-library6.0 has been built for project stackforge/fuel-library.
Files placed in repository:
fuel-ha-utils6.0-6.0.0-6205.1.noarch.rpm
fuel-library6.0-6.0.0-6205.1.noarch.rpm
Repository URL: http://osci-obs.vm.mirantis.net:82/centos-fuel-6.0-updates-stable/centos .

[OSCI Robot (oscirobot)]

Changeset merged. Package placed on primary repository.
DEB package fuel-library has been built for project stackforge/fuel-library.
Files placed in repository:
fuel-ha-utils6.0_6.0.0-6205.1_all.deb
fuel-library6.0_6.0.0-6205.1_all.deb
Repository URL: http://osci-obs.vm.mirantis.net:82/ubuntu-fuel-6.0-updates-stable/ubuntu .

[JohnsonYi (yichengli)]

This problem also occurred on ubuntu with fuel 6.0, F.Y.I

[JohnsonYi (yichengli)]

[Ubuntu]
If I reboot all 3 controller node, radosgw on the first node will not be started.

[JohnsonYi (yichengli)]

More information about upstart for radosgw
https://github.com/ceph/ceph-cookbook/issues/177

[Alex Schultz (alex-schultz)]

JohnsonYi, the fix for 6.0 was created and merged but I'm not sure on if it was released. The review for the 6.0 fix is https://review.openstack.org/#/c/200070/ which you could manually apply for your environment to address the issue.

[JohnsonYi (yichengli)]

@Alex Thanks for the information.