significant performance degradation when ceilometer middleware for swift proxy uses
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ceilometer |
Invalid
|
Low
|
Unassigned |
Bug Description
[Description]
I measured swift's performance when ceilometer middleware for swift proxy used.
The result is following:
PUT : confirmed 20-50% degradation
GET : confirmed 20-80% degradation
DELETE : seems to be no problem with the data.
I think that it is very difficult to accept these degradation.
What do you think?
[Version details]
IceHouse(swift) : 1.13.1-1.el6
swift-bench : 1.1-dev
[Crystal clear details to reproduce the bug]
swift-bench
[Test environment details]
- 4 x clients CentOS 6.5 (2.93GHz/15GB) on Xen
- 1 x Proxy Node CentOS 6.5 (2.93GHz/24GB) on Xen
- 3 x Storage Nodes CentOS 6.5 (2.93GHz/12GB) on Xen
[Actual results]
Without ceilometer middleware for swift proxy
- PUT (MB/s)
ObjectSize Concurrency
1 2 4 8 16 32
4KB 0.03 0.03 0.02 0.02 0.02 0.03
16KB 0.14 0.13 0.10 0.08 0.10 0.10
64KB 0.43 0.46 0.28 0.32 0.38 0.40
256KB 1.93 1.48 0.75 1.00 1.24 1.32
- GET (MB/s)
ObjectSize Concurrency
1 2 4 8 16 32
4KB 0.19 0.20 0.16 0.07 0.07 0.09
16KB 0.87 0.89 0.27 0.38 0.31 0.37
64KB 1.76 2.04 0.77 1.09 0.95 0.95
256KB 5.65 2.66 2.27 2.39 2.29 2.98
- DELETE (/s)
ObjectSize Concurrency
1 2 4 8 16 32
4KB 16.00 8.60 5.73 7.35 8.03 9.33
16KB 12.80 7.55 6.25 9.33 7.93 9.80
64KB 16.50 7.60 7.30 6.60 7.45 9.38
256KB 9.30 9.15 8.23 5.80 7.38 9.33
With ceilometer middleware for swift proxy
- PUT (MB/s)
ObjectSize Concurrency
1 2 4 8 16 32
4KB 0.03 0.02 0.01 0.01 0.01 0.02
16KB 0.10 0.07 0.05 0.05 0.06 0.06
64KB 0.35 0.33 0.20 0.21 0.23 0.25
256KB 1.53 1.23 0.69 0.76 0.81 0.95
- GET (MB/s)
ObjectSize Concurrency
1 2 4 8 16 32
4KB 0.06 0.04 0.02 0.03 0.03 0.03
16KB 0.23 0.18 0.11 0.11 0.12 0.14
64KB 0.91 0.68 0.39 0.44 0.43 0.48
256KB 2.65 1.95 1.27 1.33 1.59 1.70
- DELETE (/s)
ObjectSize Concurrency
1 2 4 8 16 32
4KB 14.80 9.55 5.25 6.28 6.93 8.58
16KB 16.30 10.20 5.83 6.68 7.95 9.95
64KB 12.20 9.90 5.83 6.83 7.70 8.90
256KB 8.20 7.95 5.58 6.13 7.73 7.68
[Expected results]
I have no idea but less than 10% degradation would be fine for me.
I think this problem is related to the following logic.
The "publish_sample" is included in the measurement range so it is
better to remove this method (specially communication to the bus) in
the range.
- swift_middleware.py
96 def iter_response(
...
115 self.publish_
116 input_proxy.
117 bytes_sent)
...
126 else:
127 return iter_response(
description: | updated |
Changed in ceilometer: | |
assignee: | nobody → Keisuke Yamamoto (keisuk-yamamoto) |
status: | New → In Progress |
Changed in ceilometer: | |
importance: | Undecided → High |
Changed in ceilometer: | |
milestone: | none → juno-3 |
Changed in ceilometer: | |
milestone: | juno-3 → none |
Changed in ceilometer: | |
milestone: | none → juno-rc1 |
Changed in ceilometer: | |
milestone: | juno-rc1 → none |
Changed in ceilometer: | |
assignee: | Keisuke Yamamoto (keisuk-yamamoto) → nobody |
Changed in ceilometer: | |
status: | Incomplete → Invalid |
Changing bug to ceilometer since it is a bug in the ceilometer middleware