fs quota calculation is wrong

Bug #1795919 reported by Thomas Sommer on 2018-10-03
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
dovecot (Ubuntu)
Undecided
Unassigned
Bionic
Undecided
Unassigned

Bug Description

[Impact]

 * Dovecot users using the fs quota plugin see wrong quota calculation
   (overquota situation)

 * LTS version is affected. Bug would remain for a long time.

 * Patch is simple (1 line)

[Test Case]

 * Prerequisite: user quota configured on filesystem where dovecot stores your
   email (e.g. /home partition)

 * configure dovecot fs quota plugin. for details see below.

 * use doveadm command to verify the result:
   $doveadm quota get -u user
   The result is shown in kb and as a percentage value

[Regression Potential]

 * only users using the fs quota plugin are affected

Packages:
dovecot 2.2.33.2-1-ubuntu4.1
ubuntu 18.04.1 LTS

The quota is erroneously calculated with the fs quota plugin.
It was working fine on ubuntu 16.04 with the same config.

Quota configuration:
plugin {
   quota = fs:User quota
}

Here's the IMAP Session:

* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] Dovecot (Ubuntu) ready.
? LOGIN user password
? OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE QUOTA] Logged in
? GETQUOTAROOT INBOX
* QUOTAROOT INBOX "User quota"
* QUOTA "User quota" (STORAGE 2775445504 4096001)
? OK Getquotaroot completed (0.001 + 0.000 secs).

Storage should actually be around 2775445 KB.
The quota is correct from my ext3 filesystem quota of 4096001 KB.

Output of repquota:
*** Report for user quotas on device /dev/sdc
Block grace time: 7days; Inode grace time: 7days
                        Block limits File limits
User used soft hard grace used soft hard grace
----------------------------------------------------------------------
user -- 2710396 4096001 4194304 65743 0 0

Robie Basak (racb) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better.

Unfortunately I don't see anything I can do about this in Ubuntu right now. Can you reproduce this problem with a build direct from upstream sources (of this version - 2.2.33.2), without packaging? If so, then filing a bug upstream would be appropriate. If it does not, please let us know here with full steps to reproduce on a fresh Ubuntu system and we might be able to figure out if packaging is breaking it somehow.

tags: added: needs-upstream-report
Changed in dovecot (Ubuntu):
status: New → Incomplete
Thomas Sommer (flightsupport) wrote :

Somebody already reported it to the dovecot mailing list:
https://www.dovecot.org/list/dovecot/2017-June/108423.html

It seems it's related to a glibc change.
Unfortunately there's no resolution.

Thomas Sommer (flightsupport) wrote :

I think I found the problem when looking at the source which is related to glibc.
The fs quota plugin is not compatible with the shipped glibc.

Current ubuntu packaged dovecot version v2.2.33:
File: core/src/plugins/quota/quota-fs.c
https://github.com/dovecot/core/blob/release-2.2.33/src/plugins/quota/quota-fs.c

/* Older sys/quota.h doesn't define _LINUX_QUOTA_VERSION at all, which means
   it supports only v1 quota */
#ifndef _LINUX_QUOTA_VERSION
# define _LINUX_QUOTA_VERSION 1
#endif

Current dovecot release v2.3.3
File: core/src/plugins/quota/quota-fs.c
https://github.com/dovecot/core/blob/release-2.3.3/src/plugins/quota/quota-fs.c

/* Very old sys/quota.h doesn't define _LINUX_QUOTA_VERSION at all, which means
   it supports only v1 quota. However, new sys/quota.h (glibc 2.25) removes
   support for v1 entirely and again it doesn't define it. I guess we can just
   assume v2 now, and if someone still wants v1 support they can add
   -D_LINUX_QUOTA_VERSION=1 to CFLAGS. */
#ifndef _LINUX_QUOTA_VERSION
# define _LINUX_QUOTA_VERSION 2
#endif

Ubuntu 1804 glibc is at version 2.27.

I compiled dovecot core release 2.3.3 from source.
Result: The fs quota plugin works as expected.

Any chance the package is upgraded to version 2.3.3?

On Sun, Oct 07, 2018 at 08:50:21PM -0000, Thomas Sommer wrote:
> Any chance the package is upgraded to version 2.3.3?

Not for 18.04. Please see https://wiki.ubuntu.com/StableReleaseUpdates
for rationale and policy. The easiest way to fix the quota plugin on
18.04 is to find the minimal patch that fixes it - not a wholesale
version update that could break/upset other users who expect stability.

Thomas Sommer (flightsupport) wrote :

Are you able to test the attached patch?
I wasn't able to build the package on my current system.
Or I just don't know how to do it. With apt source dovecot-core I got the sources for dovecot 2.2.22, but the installed version is 2.2.33.2

Robie Basak (racb) wrote :

I uploaded a build to my experimental PPA at: https://launchpad.net/~racb/+archive/ubuntu/experimental/

You should be able to try it from there, assuming the build succeeds.

You can use "pull-lp-source" (from ubuntu-dev-tools) to grab the Ubuntu sources for any given package and version.

Sorry it's currently rather arcane to rebuild something from Ubuntu with just a patch applied. It should be easier, and we're working on it. It's still experimental, but you can see a git view of the sources at: https://code.launchpad.net/ubuntu/+source/dovecot

The attachment "quota-fs.diff" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

tags: added: patch
Thomas Sommer (flightsupport) wrote :

The patch seems to work. Dovecot now displays the correct quota.
Also the command doveadm quota get -u user shows correct values.
What I'm unable to verify is the quota-status-service for postfix.

I have it configured in dovecot.conf as follows:
plugin {
    quota_grace = 10%%
    # 10% is the default
    quota_status_success = DUNNO
    quota_status_nouser = DUNNO
    quota_status_overquota = "552 5.2.2 Mailbox is full"
}

service quota-status {
    executable = quota-status -p postfix
    inet_listener {
        port = 8881
    }
    client_limit = 1
}

When I test it, while being clearly over quota, I get the following:

$doveadm quota get -u user
Quota name Type Value Limit %
User quota STORAGE 3901636 3300000 118

$printf "recipient=user\nsize=1230444\n\n" | nc -q1 localhost 8881
action=DUNNO

In my opinion it should return "552 5.2.2 Mailbox is full" instead of DUNNO.

Maybe somebody is able to test this?

Thomas Sommer (flightsupport) wrote :

I was away for a while, but in the meantime the patched dovecot was running all the time without any issues.
I tried to replicate the service quota-status on a freshly installed ubuntu 16.04.5. I get the same result of DUNNO instead of Mailbox full. So no regression here.

May I suggest to release a new dovecot-core package with the patch?

Thomas Sommer (flightsupport) wrote :

~racb what do you think? Could you release a new version with the patch included?

Robie Basak (racb) wrote :

Sorry I missed your previous comment. Please could you see https://wiki.ubuntu.com/StableReleaseUpdates#Procedure and handle as much of that as you can? I'll be happy to help with with the rest. Please could you also verify that this bug is fixed in Ubuntu 18.10 (Cosmic)?

Changed in dovecot (Ubuntu Bionic):
status: New → Triaged
tags: added: server-next
Robie Basak (racb) on 2018-11-27
tags: added: bitesize
Thomas Sommer (flightsupport) wrote :

I've added the SRU Bug Template. Could you take it from here?

description: updated
Andreas Hasenack (ahasenack) wrote :

https://github.com/dovecot/core/commit/39eefc53d06cf9790c6e3180a67b68e89eb71313#diff-e72982e5f369f46babc0f4b79df68878 is the upstream commit corresponding to the attached patch.

I see that this fix is present in the Ubuntu Disco and Cosmic packages, so I'm marking the main task of this bug as fix released.

Changed in dovecot (Ubuntu):
status: Incomplete → Fix Released
Thomas Sommer (flightsupport) wrote :

But should still be backported to bionic IMHO.

Robie Basak (racb) wrote :

Yes, it's on our list to do soon.

Nick Maynard (nick-maynard) wrote :

Thank you - we have just upgraded our mail system from xenial to bionic, and unexpectedly hit this.

It seems it *might* have other side effects, like breaking indexing (checks quota before writing files) and potentially writing to Sent and Drafts (also, checking quota).

High priority for us - a backport to bionic would be highly appreciated.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers