Swift Account quota-count is not honored

[Sachin]

==========================================================================
- Ensure that setting the quota is only possible by someone with reseller rights
- You need to compare account_info['meta'].get('quota-count') and account_info['count']
- Add or update the tests
============================================================================

(with reseller_admin)
[vagrant@saio swift]$ swift stat -v
                 StorageURL: http://192.168.11.33:8080/v1/AUTH_admin
                 Auth Token: AUTH_tkb9f6870204b8459abc8f09cd0f0352eb
                    Account: AUTH_admin
                 Containers: 0
                    Objects: 0
                      Bytes: 0
Containers in policy "gold": 0
   Objects in policy "gold": 0
     Bytes in policy "gold": 0
     X-Openstack-Request-Id: tx9d19c1d021da40fc9045f-00586a5b1d
                X-Timestamp: 1483363336.98908
                 X-Trans-Id: tx9d19c1d021da40fc9045f-00586a5b1d
               Content-Type: text/plain; charset=utf-8
              Accept-Ranges: bytes

[vagrant@saio swift]$ swift post -m quota-count:3
[vagrant@saio swift]$ swift stat -v
                 StorageURL: http://192.168.11.33:8080/v1/AUTH_admin
                 Auth Token: AUTH_tkb9f6870204b8459abc8f09cd0f0352eb
                    Account: AUTH_admin
                 Containers: 0
                    Objects: 0
                      Bytes: 0
Containers in policy "gold": 0
   Objects in policy "gold": 0
     Bytes in policy "gold": 0
           Meta Quota-Count: 3
                 X-Trans-Id: tx15a7738a90f449929fd87-00586a5b52
     X-Openstack-Request-Id: tx15a7738a90f449929fd87-00586a5b52
                X-Timestamp: 1483363336.98908
               Content-Type: text/plain; charset=utf-8
              Accept-Ranges: bytes

(without reseller_admin)
[vagrant@saio swift]$ swift upload keys CHANGELOG
CHANGELOG

[vagrant@saio swift]$ swift upload keys LICENSE
LICENSE

[vagrant@saio swift]$ swift upload keys AUTHORS
AUTHORS

[vagrant@saio swift]$ swift list keys --lh
 12K 2017-01-02 13:56:38 application/octet-stream AUTHORS
 81K 2017-01-02 13:56:01 application/octet-stream CHANGELOG
 11K 2017-01-02 13:56:34 application/octet-stream LICENSE
104K

[vagrant@saio swift]$ swift upload keys README.rst
README.rst
[vagrant@saio swift]$ swift list keys --lh
 12K 2017-01-02 13:56:38 application/octet-stream AUTHORS
 81K 2017-01-02 13:56:01 application/octet-stream CHANGELOG
 11K 2017-01-02 13:56:34 application/octet-stream LICENSE
5.4K 2017-01-02 13:56:52 application/octet-stream README.rst
110K

[vagrant@saio swift]$ swift stat -v
                 StorageURL: http://192.168.11.33:8080/v1/AUTH_admin
                 Auth Token: AUTH_tkb9f6870204b8459abc8f09cd0f0352eb
                    Account: AUTH_admin
                 Containers: 1
                    Objects: 4
                      Bytes: 113005
Containers in policy "gold": 1
   Objects in policy "gold": 4
     Bytes in policy "gold": 113005
           Meta Quota-Bytes: 30000
           Meta Quota-Count: 3
                 X-Trans-Id: tx905b4dc11b7b43e98130c-00586a5c39
     X-Openstack-Request-Id: tx905b4dc11b7b43e98130c-00586a5c39
                X-Timestamp: 1483363336.98908
               Content-Type: text/plain; charset=utf-8
              Accept-Ranges: bytes
[vagrant@saio swift]$ swift stat -v
                 StorageURL: http://192.168.11.33:8080/v1/AUTH_admin
                 Auth Token: AUTH_tkb9f6870204b8459abc8f09cd0f0352eb
                    Account: AUTH_admin
                 Containers: 1
                    Objects: 4
                      Bytes: 113005
Containers in policy "gold": 1
   Objects in policy "gold": 4
     Bytes in policy "gold": 113005
           Meta Quota-Bytes: 30000
           Meta Quota-Count: 3
                 X-Trans-Id: tx099981ba8cb64fdabe292-00586a5c3b
     X-Openstack-Request-Id: tx099981ba8cb64fdabe292-00586a5c3b
                X-Timestamp: 1483363336.98908
               Content-Type: text/plain; charset=utf-8
              Accept-Ranges: bytes

[Christian Schwede (cschwede)]

I couldn't verify this using the current master branch - it worked as expected in my quick test.

Can you please share some more information?

- which version did you use?
- what is the output of "swift info"?
- can you share the proxy pipeline config from /etc/swift/proxy-server.conf?

Thanks,

Christian

[Christian Schwede (cschwede)]

Just stumbled upon this:

swift post -m quota-count:3

This won't work because account quotas only limit bytes, not numbers of objects. The above command will be accepted by Swift because it's just another metadata entry, but none that will be used by the account quota middleware.

This is also nowhere documented, as far as I can tell. Do you want this to be implemented in Swift?

[Sachin (sacpatil)]

- which version did you use?
> python-swiftclient 3.2.1.dev9
> swift 2.12.0 (cbb99f97b02857dd638b0d61f5e19b584037b25f)

- what is the output of "swift info"?
> Core: swift
 Options:
  account_autocreate: True
  account_listing_limit: 10000
  allow_account_management: True
  container_listing_limit: 10000
  extra_header_count: 0
  max_account_name_length: 256
  max_container_name_length: 256
  max_file_size: 5368709122
  max_header_size: 8192
  max_meta_count: 90
  max_meta_name_length: 128
  max_meta_overall_size: 4096
  max_meta_value_length: 256
  max_object_name_length: 1024
  policies: [{u'default': True, u'name': u'gold', u'aliases': u'gold'}, {u'name': u'silver', u'aliases': u'silver'}, {u'name': u'ec42', u'aliases': u'ec42'}]
  strict_cors_mode: True
  version: 2.12.1.dev7
Additional middleware: account_quotas
Additional middleware: bulk_delete
 Options:
  max_deletes_per_request: 10000
  max_failed_deletes: 1000
Additional middleware: bulk_upload
 Options:
  max_containers_per_extraction: 10000
  max_failed_extractions: 1000
Additional middleware: container_quotas
Additional middleware: container_sync
 Options:
  realms: {u'SAIO': {u'clusters': {u'SAIO_ENDPOINT': {u'current': True}}}}
Additional middleware: crossdomain
Additional middleware: ratelimit
 Options:
  account_ratelimit: 0.0
  container_listing_ratelimits: []
  container_ratelimits: []
  max_sleep_time_seconds: 60.0
Additional middleware: slo
 Options:
  max_manifest_segments: 1000
  max_manifest_size: 2097152
  min_segment_size: 1
Additional middleware: staticweb
Additional middleware: tempauth
 Options:
  account_acls: True
Additional middleware: tempurl
 Options:
  incoming_allow_headers: []
  incoming_remove_headers: [u'x-timestamp']
  methods: [u'GET', u'HEAD', u'PUT', u'POST', u'DELETE']
  outgoing_allow_headers: [u'x-object-meta-public-*']
  outgoing_remove_headers: [u'x-object-meta-*']
Additional middleware: versioned_writes
 Options:
  allowed_flags: [u'x-versions-location', u'x-history-location']

- can you share the proxy pipeline config from /etc/swift/proxy-server.conf?
[pipeline:main]
# Yes, proxy-logging appears twice. This is so that
# middleware-originated requests get logged too.
pipeline = catch_errors gatekeeper healthcheck proxy-logging cache bulk tempurl ratelimit crossdomain container_sync tempauth staticweb copy container-quotas account-quotas slo dlo versioned_writes proxy-logging proxy-server

- swift post -m quota-count:3
This won't work because account quotas only limit bytes, not numbers of objects. The above command will be accepted by Swift because it's just another metadata entry, but none that will be used by the account quota middleware.

This is also nowhere documented, as far as I can tell. Do you want this to be implemented in Swift?

> Yeah, I though about this after looking AccountQuotaMiddleware class. Are there any plans to implement this in future? I want this to be implemented. If you give me some pointers, I can take this myself.

[Sachin (sacpatil)]

@Christian

Hope I'm not doing anything wrong..but I tried it again, I got same results

[vagrant@saio swift]$ swift post -m quota-bytes:30000

[vagrant@saio swift]$ swift stat -v
                 StorageURL: http://192.168.11.33:8080/v1/AUTH_test
                 Auth Token: AUTH_tk665649077fc74fca88eebd7274de17f4
                    Account: AUTH_test
                 Containers: 0
                    Objects: 0
                      Bytes: 0
Containers in policy "gold": 0
   Objects in policy "gold": 0
     Bytes in policy "gold": 0
           Meta Quota-Bytes: 30000
     X-Openstack-Request-Id: tx168e498aed244d9bb47b4-00586b9d5b
                X-Timestamp: 1483034243.85889
                 X-Trans-Id: tx168e498aed244d9bb47b4-00586b9d5b
               Content-Type: text/plain; charset=utf-8
              Accept-Ranges: bytes

[vagrant@saio swift]$ du -b CHANGELOG
83412 CHANGELOG

[vagrant@saio swift]$ swift upload keys CHANGELOG
CHANGELOG

[vagrant@saio swift]$ swift list keys
CHANGELOG

[vagrant@saio swift]$ swift list keys --lh
 81K 2017-01-03 12:48:19 application/octet-stream CHANGELOG
 81K

[Christian Schwede (cschwede)]

In your case you are setting the quota first, and then do a request with the same user.
However, that user must be an user with reseller rights, otherwise setting the quota would be not allowed. At the same time a user with reseller rights will always bypass the quota check. This is why you can exceed the quota.

Have a look at these lines:

https://github.com/openstack/swift/blob/master/swift/common/middleware/account_quotas.py#L96-L99

Normally you would set the quota using an account with reseller rights, but use a regular admin or user without reseller rights to store data in Swift - and in that case the quota will be enforced.

So if you retry storing data by an user without reseller rights, this should fail if the quota is exceeded.

> Are there any plans to implement this in future? I want this to be implemented.
> If you give me some pointers, I can take this myself.

There are no plans afaik, so feel free to start hacking :)
Should be pretty straightforward:

- ensure that setting the quota is only possible by someone with reseller rights
- you need to compare account_info['meta'].get('quota-count') and account_info['count']
- add or update the tests

[Sachin (sacpatil)]

Ahh! That was my mistake. I was trying to upload an object with reseller_admin rights. Didn't noticed that quota is bypassed for reseller_admin in the code.

I will try to implement quota-count to account and change the subject of this bug accordingly.

Thanks a lot Christian!

[Janie Richling (jrichli)]

FYI: an older patch https://review.openstack.org/#/c/312603/ Adds support for object count quotas at a storage account level.

[Sachin (sacpatil)]

@jrichli: I'll go through the patch. Thanks for the info.