performance of accounts-daemon is very poor

[Peter Gray]

Version 11.10

The performance of accounts-daemon is unacceptable with large password and/or group files. We have appoximately 40000 users in the passwords file and 16000 lines in the group file. Having looked at the source, it appears to me the problem is that for each user is pulls from the password database (via getpwXXX) its then calls getgroups to return the users list of groups.

Getgroups must make a complete pass through the group file to determine the groups for the user. When initializing this results in
reading the groups file 40000 times.

I think a better solution is to build a inverted group file map then first time getgroups is called by reading the entire group database and building a data structure indexed by username where a users group list can by found in a single lookup.
Of couse, an even better solution would be for getgroups to do that itself, but that would require much larger code changes.

I tried installed nscd to see if this would make a difference but no dice.

At the moment, 11.10 is unusable for us because of this problem.

Hope this is all the info you need.

Regards, pdg

<email address hidden>

[Peter Gray (pdg-o)]

Sorry, I said getgroups in my original post, and I should have said getgrouplist.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Sebastien Bacher (seb128)]

Thank you for your bug report, that seems like the sort of issues that should be discussed upstream, somebody should report a bug on https://bugs.freedesktop.org/enter_bug.cgi?product=accountsservice

While the issue is an important one it's limited to some big deployments and those can probably uninstall accountsservices (lightdm will fallback if it's not sure), ideally we would work on the issue but we have limited resources and it's not on the priority list

[Vadim Rutkovsky (roignac)]

Original report: https://bugs.launchpad.net/bugs/941673

The performance of accounts-daemon is unacceptable with large password and/or group files. We have appoximately 40000 users in the passwords file and 16000 lines in the group file. Having looked at the source, it appears to me the problem is that for each user is pulls from the password database (via getpwXXX) its then calls getgrouplist to return the users list of groups.

Getgroups must make a complete pass through the group file to determine the groups for the user. When initializing this results in
reading the groups file 40000 times.

I think a better solution is to build a inverted group file map then first time getgroups is called by reading the entire group database and building a data structure indexed by username where a users group list can by found in a single lookup.
Of couse, an even better solution would be for getgroups to do that itself, but that would require much larger code changes.

I tried installed nscd to see if this would make a difference but no
dice.

[Vadim Rutkovsky (roignac)]

Thank you for your bug report. This bug has been reported to the developers of the software. You can track it and make comments at https://bugs.freedesktop.org/show_bug.cgi?id=48177

[Rstrode (rstrode)]

AccountsService sort of assumes users in /etc/passwd are "local users" and that the 40000 user network login cases are covered by nis/ldap/etc

I guess we should do some sort of sanity check and treat /etc/passwd users as remote in the case the check fails. This way we sidestep all the costly work entirely.

[Rstrode (rstrode)]

maybe we could just bail if there are greater than N found users. The problem then is finding a suitable value of N.

[Leonardo Borda (lborda)]

Hi Guys,

As a workaround for the time being one could do the following (tested on 12.04):

Remove accountsservice
sudo apt-get remove accountsservice

Edit /etc/lightdm/lightdm.conf and add the following line :
greeter-hide-users=true

Then restart lightdm or reboot the machine:
sudo service lightdm restart

Leonardo Borda

[Julian Clark (julian.clark)]

My setup is Ubuntu 12.04 with Gnome + Compiz in use instead of Unity.

I am not quite confident with the suggested workaround. The package "accountsservice" appears to be a key dependency for some seemingly important packages. In my case, those packages are "gnome-control-center," "language-selector-common," and "language-selector-gnome." The package "ubuntu-desktop" (which is a commonly-installed package) has "gnome-control-center" as a dependency. I cannot see how uninstalling "accountsservice" will not significantly reduce overall functionality and/or break something.

[Peter Gray (pdg-o)]

FYI,

I think I have a workaround for this problem by putting

UID_MIN 9999999

into /etc/gdm/custom.conf

This seems fragile since I am not sure the interaction of accountsservices and custom.conf is stable or documented but it appears to solve the problem. Of course, this is not a general solution for all sites, but happens to work for me.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "a temporary hackish workaround/patch" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-reviewers team please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

[Ritesh Khadgaray (khadgaray)]

Created attachment 65941
proposed patch

limit the no of user listed by a-s to UID_MAX. Also, sets user-limit to true when this condition is meet, allowing apps like gdm/lightdm to enable manual login option.

[Rstrode (rstrode)]

Talked with Ritesh about this bug today on IRC. He's going to look into using fgetgrent to create the hash table of local groups and use that.

[Ritesh Khadgaray (khadgaray)]

Created attachment 66171
proposed patch

[Rstrode (rstrode)]

ritz, any chance you could get this applying against master (preferably as a git-format-patch formatted patch) ?

[Tatsis Dimitrios (dimitrist16)]

Our setup is affected by this bug when the ldap server is offline. What is the proper way to disable accounts service and 'fallback lightdm' ? I removed accountsservice and everything works fine except that a handful of packets depend on it (Ubuntu 10.04)

* The following packages will be REMOVED:
* accountsservice gdm gnome-control-center indicator-datetime indicator-power language-selector-common
* language-selector-gnome ubuntu-desktop ubuntu-standard

I really don't see why all these dependencies exist. Isn't there supposed to be an alternative ?

[Rstrode (rstrode)]

any updates on this?

[Peter Gray (pdg-o)]

On Fri, Jan 04, 2013 at 04:20:15PM -0000, Rstrode wrote:
> any updates on this?

Errr... I am not working on this. I just reported it.

I managed a work around so the problem went away for me.

Regards,
pdg

[Ritesh Khadgaray (khadgaray)]

Created attachment 73763
patch

Hi halfline

  I have rewritten the code to use hashtable instead of list, leading to lot faster startup time on my test system ( <1s for ~40k users). I am currently testing the code ( add/remove users option ). This is for an older version of a-s (0.6.15) , will update the code against head and post.

  Anything which does need the user list ( gnome-control-center user manager/ indicator-session/..) still breaks .

  Personally I believe, the best solution would be to allow user to select an upper limit ( and list only cached users from login history ), and broadcast "user-limit-reached" option. The broadcast could be used by applications to display warning or change their behaviour ( such as disable user list in gdm ).

[Ritesh Khadgaray (khadgaray)]

Created attachment 74177
patch

initial patch, against latest release.

still need to work quirk against entry_generator_wtmp. It calls getpwnam for every entry ( even if it is a duplicate entry ).

[Rstrode (rstrode)]

any chance you could update this again for git master?

[Rolf Leggewie (r0lf)]

oneiric has seen the end of its life and is no longer receiving any updates. Marking the oneiric task for this ticket as "Won't Fix".