(needs x11 protocol update) Xorg needs client limit raised

Bug #260138 reported by Kees Cook on 2008-08-21
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
X.Org X server
Confirmed
Medium
x11proto-core (Ubuntu)
Wishlist
Unassigned
xorg-server (Ubuntu)
Wishlist
Unassigned

Bug Description

In include/misc.h, the current client limit is 256. This is starting to become a limiting factor in some high-RAM desktop environments. Perhaps it should be raised to 1024, as seen here:

http://www.nabble.com/Maximum-Number-of-Clients-td7307396.html

There is a 2 year old Xorg upstream bug about this as well:

https://bugs.freedesktop.org/show_bug.cgi?id=9209

I'm unclear if this change would break the ABI, though.

Related branches

Sorry about the phenomenal bug spam, guys. Adding xorg-team@ to the QA contact so bugs don't get lost in future.

I believe the problem is still in 7.2 and 7.3. RedHat has provided me a patch for 6.8.2 as a part of RHEL 4 Update 6.

In include/misc.h, the current client limit is 256. This is starting to become a limiting factor in some high-RAM desktop environments. Perhaps it should be raised to 1024, as seen here:

http://www.nabble.com/Maximum-Number-of-Clients-td7307396.html

There is a 2 year old Xorg upstream bug about this as well:

https://bugs.freedesktop.org/show_bug.cgi?id=9209

I'm unclear if this change would break the ABI, though.

Kees Cook (kees) wrote :
Kees Cook (kees) wrote :
Changed in xorg-server:
status: Unknown → Confirmed
Timo Aaltonen (tjaalton) wrote :

asked upstream about it. The patches are indeed in RHEL4, so they should be tested by now. They don't seem to qualify as-is to upstream though:

20:52 < ajax> you trade more clients for fewer resources per client
21:05 < ajax> the XID space is carved up weird
21:06 < ajax> (when i say "resources" i mean "protocol objects with an XID")
21:06 < ajax> top three bits are reserved (f*** you lisp), next eight bits are client id, the
              rest are for the client to allocate however it likes.
21:08 < ajax> so, with 256 clients, each of them can create up to 2 ^ (32 - 8 - 3) ~= 2
              million resources
21:08 < ajax> 512 clients, one million resources each. etc.
21:09 < ajax> XIDs are allocated client-side to avoid round-tripping to the server on every
              resource creation, but that means you have to be able to allocate unique ids
              within each client. so you have to have some partition...
21:10 < ajax> i suppose the right way to fix this is to do a field survey of how many
              resources a typical "small" app and a typical "big" app use.
21:13 < ajax> on my machine, metacity is far and away the one with the most resources, and
              it's still down around the 2000 mark
21:13 < ajax> you do still want a fairly sizable allocation per client since XID reuse is
              pretty fast and furious, and requires a round trip through xc-misc
21:14 < ajax> and, you can't know that you're talking to an xlib that's new enough to know
              about the newer version of xc-misc
21:18 < ajax> this _might_ be a case where you could do protocol 11.1 meaningfully
21:18 < ajax> which would let the server assign huge ranges to 11.0 clients and tiny ones to
              11.0 clients
21:18 < ajax> uh, tiny ones to 11.1 clients

so, we can fix this in our packages, but it would not be the final solution.

Changed in xorg-server:
importance: Undecided → Wishlist
status: New → Confirmed
Changed in x11proto-core:
importance: Undecided → Wishlist
status: New → Confirmed

On Thu, Aug 21, 2008 at 06:33:17PM -0000, Timo Aaltonen wrote:
> 21:08 < ajax> so, with 256 clients, each of them can create up to 2 ^ (32 - 8 - 3) ~= 2
> million resources
> 21:08 < ajax> 512 clients, one million resources each. etc.

Sounds like bumping to 512 would be less of a resource hit than to 1024.
How do you query the number of resources an application uses? (I have
been using "xlsclients" to get a rough estimate of the number of
clients.)

xrestop, then add Wins, GCs, Fnts, Pxms and Misc together. For me, firefox gives around 3500.

I agree that 512 might be a good compromise.

Bryce Harrington (bryce) wrote :

512 sounds good to me as well, however I'm a bit uncomfortable with doing this patch post-featurefreeze. Could this be an Intrepid+1 thing?

Changed in xorg-server:
status: Confirmed → Triaged
Changed in x11proto-core:
status: Confirmed → Triaged
Kees Cook (kees) on 2008-11-21
Changed in x11proto-core:
milestone: none → jaunty-alpha-4
Bryce Harrington (bryce) wrote :

Plan is to tackle this post-alpha3, as it will require changing x11proto-core first so will be safest to do post-release.

Changed in xorg-server:
status: Triaged → In Progress
Bryce Harrington (bryce) on 2009-01-27
Changed in xorg-server:
assignee: nobody → bryceharrington
Bryce Harrington (bryce) wrote :

<tjaalton> bryce-sprint: well, the proto patch uses 16bits instead of 8, so maybe 9 would do for 512, but I guess 16 doesn't hurt. the one for xorg-server is trivial

Bryce Harrington (bryce) wrote :

Please review. For the proto patch I changed to 9 instead of 16 - is this correct?

Bryce Harrington (bryce) wrote :

scratch the last one

Bryce Harrington (bryce) wrote :

Patch committed to git; will be uploaded after freeze is lifted.

Changed in xorg-server:
status: In Progress → Fix Committed
Bryce Harrington (bryce) wrote :

Patch for x11proto-core is ready to go once archive unfrozen.

Changed in x11proto-core:
status: Triaged → In Progress
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package xorg-server - 2:1.5.99.902-0ubuntu2

---------------
xorg-server (2:1.5.99.902-0ubuntu2) jaunty; urgency=low

  [Bryce Harrington]
  * Add 157_check_null_modes.patch: Catch null pointer dereference in
    video mode selection, which can cause xserver crash when using SDL
    applications with qemu/kvm.
    (LP: #300310)
  * Add 158_raise_maxclients.patch to raise max number of clients from 256
    to 512. Trade-off is that this reduces client resources available to
    1,048,576 total resources (which should still be ample).
    (LP: #260138)

  [Steven Harms]
  * 159_xinerama_focus.patch: Resolves xinerama focus issues
    with multiple screens
    (LP: #41301)

 -- Bryce Harrington <email address hidden> Wed, 04 Feb 2009 22:33:28 -0800

Changed in xorg-server:
status: Fix Committed → Fix Released
Hew McLachlan (hew) wrote :

The xorg-server patch has caused bug 326344.

Bryce Harrington (bryce) wrote :

Reverted patch, since it breaks -nvidia.

Sorry, it seems to do this properly requires a rev of the X11 protocol, which is a much bigger endeavor than can be done in Ubuntu. Upstream seems open to doing this though, but as this is a wishlist item, I would prefer you would please work with upstream (ajax) directly on this.

Changed in xorg-server:
status: Fix Released → Triaged
Changed in x11proto-core:
status: In Progress → Triaged
Changed in xorg-server:
assignee: bryceharrington → nobody

Note that raising MAXCLIENTS lowers the number of resources each client
can have since the resource namespace is fixed across all clients in the
X11 protocol, and evenly divided among them.

Years ago we fixed this in Xsun in Solaris by offering an option to set
maxclients at runtime to either 128 or 1024, allowing sites the option
to set this themselves, but it did incur a small performance cost across
the X server everytime that flag was checked. If someone wanted to
investigate a similar option in Xorg, I could release that code.

(In reply to comment #3)
> Note that raising MAXCLIENTS lowers the number of resources each client
> can have since the resource namespace is fixed across all clients in the
> X11 protocol, and evenly divided among them.
>
> Years ago we fixed this in Xsun in Solaris by offering an option to set
> maxclients at runtime to either 128 or 1024, allowing sites the option
> to set this themselves, but it did incur a small performance cost across
> the X server everytime that flag was checked. If someone wanted to
> investigate a similar option in Xorg, I could release that code.

It is clear that the tradeoff that was made to only allow 256 connections is heavily lopsided towards allowing those clients to open far more resources than any actually use in practice.

https://bugs.launchpad.net/ubuntu/+source/xorg-server/+bug/260138

Large clients have been shown to use 3500 resources each, but we allow them to use 2^29/256, or 2 million. Can't we steal quite a few more bits for increasing the number of X clients? I don't just want a doubling - I never come close to the resources-per-client limit, but I am always running into the number-of-clients limit.

If it's a bit too much work to add a configuration or cmdline option (your patch would definitely be worthwhile though), can't we change the default (such as in http://readlist.com/lists/lists.freedesktop.org/xorg/3/17055.html ) anyway, since it's currently so heavily lopsided?

(In reply to comment #4)
> It is clear that the tradeoff that was made to only allow 256 connections is
> heavily lopsided towards allowing those clients to open far more resources than
> any actually use in practice.

Than any your users use perhaps - we have had users with applications that ran
out, especially CAD/CAM type applications. Remember that older toolkits did
a lot more server side than GTK & Qt today, and some did things like creating
a new Window id for every widget.

> If it's a bit too much work to add a configuration or cmdline option (your
> patch would definitely be worthwhile though), can't we change the default (such
> as in http://readlist.com/lists/lists.freedesktop.org/xorg/3/17055.html )
> anyway, since it's currently so heavily lopsided?

While I've not extracted the change into a patch that could be easily ported/applied to Xorg, the code to the Solaris Xsun fork of the X11R6.*
server fork has since been published if someone else wants to try.

The code to the entire server is published at:
http://dlc.sun.com/osol/x/downloads/openXsun/

The command-line selectable 128/1024 clients code changes should be in these
files (and possibly more, I've not searched, just referring to our original
code changes - and obviously not all these apply to X11R7.x/Xorg 1.x):
  xc/programs/Xserver/Xext/mbuf.c
  xc/programs/Xserver/Xext/multibuf.c
  xc/programs/Xserver/Xext/security.c
  xc/programs/Xserver/Xi/closedev.c
  xc/programs/Xserver/dbe/dbe.c
  xc/programs/Xserver/dix/colormap.c
  xc/programs/Xserver/dix/dixutils.c
  xc/programs/Xserver/dix/grabs.c
  xc/programs/Xserver/include/misc.h
  xc/programs/Xserver/include/resource.h
  xc/programs/Xserver/include/windowstr.h
  xc/programs/Xserver/lbx/lbxcmap.c
  xc/programs/Xserver/lbx/lbxmain.c
  xc/programs/Xserver/lbx/lbxserve.h
  xc/programs/Xserver/os/WaitFor.c
  xc/programs/Xserver/os/connection.c
  xc/programs/Xserver/os/io.c
  xc/programs/Xserver/os/osdep.h
  xc/programs/Xserver/os/osinit.c
  xc/programs/Xserver/os/utils.c
  xc/programs/Xserver/os/xdmcp.c
  xc/programs/Xserver/pandix/dispatch.c
  xc/programs/Xserver/pandix/events.c
  xc/programs/Xserver/pandix/main.c
  xc/programs/Xserver/pandix/resource.c
  xc/programs/Xserver/pandix/window.c
  xc/programs/Xserver/record/record.c

plus at least this patch to x11proto:
http://src.opensolaris.org/source/xref/x-cons/XW_NV_tw-clone/open-src/proto/x11proto/Xpoll.h-patch

Changed in xorg-server:
importance: Unknown → Medium
Changed in xorg-server:
importance: Medium → Unknown
Changed in xorg-server:
importance: Unknown → Medium

Any progress on that? I have similar problem in Ubuntu 10.04.

Timo Aaltonen (tjaalton) wrote :

This is a risky change that is not landing upstream, therefore we should not be doing it either. The issue will fix itself after we've switched to wayland at some point in the future..

Changed in x11proto-core (Ubuntu):
milestone: jaunty-alpha-4 → none
status: Triaged → Won't Fix
Changed in xorg-server (Ubuntu):
status: Triaged → Won't Fix

This is only going to get more embarassing, as machines get more ram and toolkits continue their trend to just send a pixmap to the server. I'm hitting the 256 limit everyday now, on a meduim-spec work laptop running kde.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.