[library] Command exceeded timeout at /etc/puppet/modules/ceph/manifests/osd.pp:33

Bug #1304268 reported by Anastasia Palkina
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
Low
Fuel Library (Deprecated)
5.0.x
Won't Fix
Low
Fuel Library (Deprecated)

Bug Description

"build_id": "2014-04-07_04-17-26",
"mirantis": "yes",
"build_number": "270",
"nailgun_sha": "7a05e365240ab27c492b20585ef8ac8557102cc0",
"ostf_sha": "de0222fed646525d248dc6892eeceab139d5c469",
"fuelmain_sha": "5714f2a3cf56ebdc124e9555dfff0fe274e84bdd",
"astute_sha": "55df06b2e84fa5d71a1cc0e78dbccab5db29d968",
"release": "4.1",
"fuellib_sha": "52e7f57695f33bafa5d84d524d77f1bc3a2289b2"

1. Create new environment (CentOS, simple mode)
2. Choose GRE neutron
3. Choose both Ceph and installing ceilometer
4. Add controller, compute and 3 ceph
5. Start deployment. It was successful
6. But there is error in puppet logs on first ceph node:

(/Stage[main]/Ceph::Osd/Exec[ceph-deploy osd activate]/returns) change from notrun to 0 failed: Command exceeded timeout at /etc/puppet/modules/ceph/manifests/osd.pp:33

But ceph is alive
[root@node-15 ~]# ceph osd tree
# id weight type name up/down reweight
-1 0.36 root default
-2 0.12 host node-18
0 0.06 osd.0 up 1
4 0.06 osd.4 up 1
-3 0.12 host node-17
1 0.06 osd.1 up 1
3 0.06 osd.3 up 1
-4 0.12 host node-19
2 0.06 osd.2 up 1
5 0.06 osd.5 up 1

[root@node-15 ~]# ceph -s
  cluster bc868fd3-d7ac-4e9c-ab62-c3f162925192
   health HEALTH_OK
   monmap e1: 1 mons at {node-15=192.168.0.6:6789/0}, election epoch 2, quorum 0 node-15
   osdmap e58: 6 osds: 6 up, 6 in
    pgmap v114: 492 pgs: 492 active+clean; 14464 KB data, 12549 MB used, 368 GB / 380 GB avail
   mdsmap e1: 0/0/1 up

Revision history for this message
Anastasia Palkina (apalkina) wrote :
Andrew Woodward (xarses)
tags: added: ceph
Changed in fuel:
status: New → Confirmed
milestone: 4.1.1 → 5.0
tags: added: backports-4.1.1
Revision history for this message
Andrew Woodward (xarses) wrote :

This happens on occasion and is likely the result of the older version of ceph-deploy. We should update the version of ceph-deploy or add retries = number of osd's on this system. Puppet will re-run and pass on the next iteration.

Changed in fuel:
status: Confirmed → Triaged
Changed in fuel:
milestone: 5.0 → 5.1
Revision history for this message
Anastasia Palkina (apalkina) wrote :

Reproduced for version 5.0.1

"build_id": "2014-07-08_13-57-45",
"mirantis": "yes",
"build_number": "107",
"ostf_sha": "09b6bccf7d476771ac859bb3c76c9ebec9da9e1f",
"nailgun_sha": "c0082e3a0e8544bad7bd45c15c5dd8632ea045b5",
"production": "docker",
"api": "1.0",
"fuelmain_sha": "b0f5151d12751b9b55dcd69bd1445d0d480012d6",
"astute_sha": "a4edb51661f50c66e247e0b8d00f2d01e0658fe6",
"release": "5.0.1",
"fuellib_sha": "d4cb36208efaf51a7c0ca012fa63d596d4ee2e29"

1. Create new environment (CentOS, simple mode)
2. Choose both Ceph
3. Choose Ceilometer
4. Add controller, compute+cinder+ceph, cinder+mongo, ceph
5. Start deployment. It was successful
6. But there is error in puppet.log on node with single ceph (node-12):

2014-07-08 14:00:31 ERR

 (/Stage[main]/Ceph::Osd/Exec[ceph-deploy osd activate]/returns) change from notrun to 0 failed: Command exceeded timeout

Ceph is alive:

[root@node-9 ~]# ceph osd tree
# id weight type name up/down reweight
-1 0.24 root default
-2 0.12 host node-10
0 0.06 osd.0 up 1
1 0.06 osd.1 up 1
-3 0.12 host node-12
2 0.06 osd.2 up 1
3 0.06 osd.3 up 1

[root@node-9 ~]# ceph -s
  cluster dd4ceee6-0ce0-4ae1-86b0-d6ae9ad10d99
   health HEALTH_OK
   monmap e1: 1 mons at {node-9=192.168.0.2:6789/0}, election epoch 1, quorum 0 node-9
   osdmap e35: 4 osds: 4 up, 4 in
    pgmap v71: 960 pgs: 960 active+clean; 14464 KB data, 8367 MB used, 245 GB / 253 GB avail; 112B/s rd, 0op/s
   mdsmap e1: 0/0/1 up

Revision history for this message
Anastasia Palkina (apalkina) wrote :
Dmitry Ilyin (idv1985)
summary: - Command exceeded timeout at /etc/puppet/modules/ceph/manifests/osd.pp:33
+ [library] Command exceeded timeout at
+ /etc/puppet/modules/ceph/manifests/osd.pp:33
Changed in fuel:
milestone: 5.1 → 6.0
Revision history for this message
Dmitry Borodaenko (angdraug) wrote :

We now have a more recent ceph-deploy 1.5.9 in 5.1 so this problem should disappear. Since it's a low priority bug it will not be backported to stable release series.

Changed in fuel:
milestone: 6.0 → 5.1
status: Triaged → Fix Committed
Revision history for this message
Anastasia Palkina (apalkina) wrote :

Verified on ISO #436
"build_id": "2014-08-14_02-01-17",
"ostf_sha": "d2a894d228c1f3c22595a77f04b1e00d09d8e463",
"build_number": "436",
"auth_required": true,
"api": "1.0",
"nailgun_sha": "b5bdd19c2dbeb26ce3bd88270d09f5e7541a3aea",
"production": "docker",
"fuelmain_sha": "9f327045cdd72d406d89063393a499635be5e3d4",
"astute_sha": "b52910642d6de941444901b0f20e95ebbcb2b2e9",
"feature_groups": ["mirantis"],
"release": "5.1",
"fuellib_sha": "4b085bfbf7be973f0aa29d9d5e4f3ebd5bf789a1"

Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.