Low datetime precision leads to inconsistent order of shares
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Shared File Systems Service (Manila) |
Fix Released
|
Medium
|
Dmitry Galkin |
Bug Description
Hi,
We have recently migrated Manila from Postgres to MariaDB and the side-effect of this migration is that the created_at | updated_at | deleted_at columns of all objects have only up-to-1 second precision.
In some big projects we have now multiple shares that have exactly the same created_at timestamp. Which, in turn, means we get inconsistent order of shares when those are listed via API:
Example Shares with the same timestamp:
MariaDB [manila]> SELECT created_at FROM shares GROUP BY created_at HAVING COUNT(*) >1;
+------
| created_at |
+------
| 2018-06-05 10:35:29 |
| 2018-06-07 15:22:40 |
| 2018-06-21 05:24:33 |
| 2018-06-26 10:14:05 |
...
| 2020-01-13 13:46:48 |
| 2020-01-13 13:49:25 |
+------
1713 rows in set (0.011 sec)
MariaDB [manila]> show columns from shares;
+------
| Field | Type | Null | Key | Default | Extra |
+------
| created_at | datetime | YES | | NULL | |
| updated_at | datetime | YES | | NULL | |
| deleted_at | datetime | YES | | NULL | |
...
The list sort_key is 'created_at' by default: https:/
-> that results that two shares with the same 'created_at' timestamp will often "flip" their positions in the list between two subsequent API calls with the same parameters.
This might not be a big issue at first glance, but when used with "--limit" that causes problems, since a share might "get lost" from the list if it is located on the edge of the limited list:
Sample script to reproduce:
$ for i in {01..10}; do curl -sH "X-Auth-Token: ${OS_AUTH_TOKEN}" https:/
Easy to see by the diff:
(shares were not created or deleted during the test)
$ diff -u5 pass01.txt pass02.txt
--- pass01.txt 2019-12-11 14:59:15.089967267 +0100
+++ pass02.txt 2019-12-11 14:59:17.026663666 +0100
@@ -1,10 +1,10 @@
2a27b629-
5e4cd8bd-
613de97d-
1b1a5309-
-4ae1c17a-
774af312-
+4ae1c17a-
fb7e5dde-
-0ac73939-
-0c3a57f4-
20889446-
+158e73c2-
+0c3a57f4-
The workaround I did so far is altering the created_
Name testshare
Description
ID 589e24a8-
....
Created At 2020-01-
The respective DB migration is attached as a file. Can be easily extended to all manila objects. Possible alternative is to change the default sort key to id, but this is an API change.
Please share your thoughts.
tags: | added: api db mysql |
Changed in manila: | |
assignee: | Dmitry Galkin (galkindmitrii) → Goutham Pacha Ravi (gouthamr) |
Changed in manila: | |
assignee: | Goutham Pacha Ravi (gouthamr) → Dmitry Galkin (galkindmitrii) |
milestone: | none → wallaby-1 |
Changed in manila: | |
milestone: | wallaby-1 → wallaby-2 |
Changed in manila: | |
milestone: | wallaby-2 → wallaby-3 |
tags: | added: wallaby-rc-bugsquash |
Changed in manila: | |
status: | In Progress → Fix Released |
milestone: | wallaby-3 → wallaby-rc1 |
Hi Dmitry,
Thank you so much for reporting this bug! We did take a look at this a while ago, but for some reason we haven't updated this bug with the triage [1]
I think your fix is appropriate; would you be kind to propose the patch?
[1] http:// eavesdrop. openstack. org/meetings/ manila/ 2020/manila. 2020-01- 23-15.01. log.html# l-162