HA deployment fails with swift ring builder rebalance timeout
Bug #1260206 reported by
Vladimir Kuklin
This bug affects 1 person
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Fuel for OpenStack |
Fix Released
|
Critical
|
Andrey Korolyov |
Bug Description
Looking into logs we see that HA deployment fails with timeout of swift rebalance, which has never happened earlier.
Changed in fuel: | |
importance: | High → Medium |
importance: | Medium → Critical |
status: | New → Triaged |
assignee: | nobody → Andrey Korolyov (xdeller) |
Changed in fuel: | |
status: | Fix Committed → Fix Released |
To post a comment you must log in.
rebalance container failed *immediately* w/ expiration:
2013-12- 12T02:42: 55.037912+ 00:00 debug: (Exec[rebalance _container] (provider= posix)) Executing 'swift-ring-builder /etc/swift/ container. builder rebalance' 12T02:42: 55.038569+ 00:00 debug: Executing 'swift-ring-builder /etc/swift/ container. builder rebalance' 12T02:47: 55.187447+ 00:00 err: (/Stage[ main]/Swift: :Ringbuilder/ Swift:: Ringbuilder: :Rebalance[ container] /Exec[rebalance _container] /returns) change from notrun to 0 1 failed: Command exceeded timeout at /etc/puppet/ modules/ swift/manifests /ringbuilder/ rebalance. pp:32
2013-12-
2013-12-
buider rebalance took exactly the timeout interval to fail: 12T02:48: 04.508318+ 00:00 debug: Executing 'swift-ring-builder /etc/swift/ account. builder rebalance' 12T02:53: 04.662572+ 00:00 err: (/Stage[ main]/Swift: :Ringbuilder/ Swift:: Ringbuilder: :Rebalance[ account] /Exec[rebalance _account] /returns) change from notrun to 0 1 failed: Command exceeded timeout at /etc/puppet/ modules/ swift/manifests /ringbuilder/ rebalance. pp:32
2013-12-
2013-12-
Looks like swift issue but logs are not enough to determine the cause. Please reproduce and give me an env.