ceilomiddleware.swift ignore the swift_source in environ

[Rohit Jaiswal]

swift_source indicates that the request originated inside Swift as a side-effect handling another request. Any non-empty string means its type. Values are usually short uppercase e.g., SW for StaticWeb, FP for FormPOST. A log with an empty string means the request being logged originated from an end user.

This means it double counts many user requests. If it’s more than 10 minutes since the last request, it’s probable that swift-proxy will perform a HEAD /v1/a1, then a HEAD /v1/a/c and *then* the GET /v1/a/c/o. So three messages are emitted. To the end user, it looks like they are being charged for three requests instead of one.

An example:
- User makes a GET request to /v1/a1/c1/
- Swift(swift-proxy) does a HEAD on /v1/a1 with swift_source=GET_INFO
- then GET on /v1/a1/c1 with swift_source=GET_INFO
- then does GET/v1/a1/c1/index.html with swift_source=SW
- returns the content of the index.html file and then logs the request as:
- GET /v1/a1/c1 , with swift_source=None

So, 4 records in the log, only the final one should be metered and billable

[gordon chung (chungg)]

i don't quite understand this, we generate (up to) 3 different items per request:

1. a request event
2. a incoming.bytes meter
3. a outgoing.bytes meter

are all three being captured 4 times? or just a subset?

[Rohit Jaiswal (rohit-jaiswal-3)]

I think its the request event- storage.api.request is being captured more than once for a single PUT or POST request.

[Rohit Jaiswal (rohit-jaiswal-3)]

Tested with swift enabled on devstack and saw HEAD events getting metered in ceilometer

[Rohit Jaiswal (rohit-jaiswal-3)]

Depending on where ceilometer middleware is placed on swift proxy server WSGI pipeline, it might receive some requests that originate from other WSGI middleware that are to the left of where ceilometer is on the pipeline. These requests will have swift.source set to indicate that requests originated in milldeware as side effect of serving a client request.

ceilometer middleware should not count those as valid user requests.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/216903

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/216903
Committed: https://git.openstack.org/cgit/openstack/ceilometermiddleware/commit/?id=7005d98f47be809243d627cee7ca594390cddec2
Submitter: Jenkins
Branch: master

commit 7005d98f47be809243d627cee7ca594390cddec2
Author: Rohit Jaiswal <email address hidden>
Date: Tue Aug 25 23:30:18 2015 +0000

    Ignore request with swift.source in env

    swift.source in env indicates request
    that originate in swift middleware as a side
    effect of another client request.

    These requests should not be metered, this
    fix adds a check for swift.source in env
    and avoids generating a metering event.

    Change-Id: I2ecbde8667cc366b192a2248abc93d0936956978
    Closes-bug: 1485711

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/ceilometermiddleware 0.3.0 release.