Dithering not enabled causing gradient banding on Macbook Pro, until suspend/resume [M56P Mobility X1600]

[Jordan]

Binary package hint: xserver-xorg-video-radeon

When running Intrepid on Macbook Pro there is very clear banding in gradients including things like the heron desktop background. This seems to be caused by dithering not being enabled. Oddly, when I suspend then resume dithering becomes enabled and everything looks fine until I restart X.

lsb_release -rd:
Description: Ubuntu intrepid (development branch)
Release: 8.10

xserver-xorg-video-radeon version:
1:6.9.0+git20080802.1f3eee36-1ubuntu1

(--) PCI:*(0@1:0:0) ATI Technologies Inc M56P [Radeon Mobility X1600] rev 0, Mem @ 0x80000000/0, 0x98300000/0, I/O @ 0x00003000/0, BIOS @ 0x????????/131072

[Bryce Harrington (bryce)]

Please attach your /var/log/Xorg.0.log and a photo or screenshot that illustrates the problem.

[Jordan (jordanu)]

A screenshot will be full 24 bit color even though that's not what is being displayed ( I have already tried ) and I don't have a digital camera at the moment but I will try to borrow one soon.

[Bryce Harrington (bryce)]

Okay thanks

[Bryce Harrington (bryce)]

Here is an example of banding that I see on a different system (see https://bugs.freedesktop.org/show_bug.cgi?id=17094). Is your banding identical to this banding?

And yes, I also see that if I take a screenshot and look at it on another system, I don't see the banding.

[Jordan (jordanu)]

Yes, that is exactly what it looks like on my system.

[Bryce Harrington (bryce)]

1002 ATI Technologies Inc
        71c5 M56P [Radeon Mobility X1600]
  106b 0080 MacBook Pro

This is an RV530 family chipset. The fix I put in was for RV620, RC635, RS780, and RV770 so likely won't work here.

[Bryce Harrington (bryce)]

Hi Jordan,

Like I mentioned, the patch I did was specific to hardware that's a bit different than yours, so won't fix the issue. However, it's likely that a similar patch may be possible that's particular to your hardware. However, to begin with we need to collect some hardware about particulars about your graphics card. I've prepared a debug -ati package that will print out the info I need. Can you please install this, restart X, and then collect and attach your /var/log/Xorg.0.log? It will have lines prepended by 'BWH: ' in them, which is the info I need.

http://bryceharrington.org/ubuntu/AtiDithering/

I've built both amd64 and i386 debs, and included -dbg packages for each.

Once we have the data, I can craft a dithering-enablement patch, however my experience with the other dithering patch was that it took a lot of bit fiddling to get the right settings. Do you know how to build .deb packages yourself? If so, I could point you to the flags and how to tweak them, so you can tinker with them directly. If not, I can make some guesses and we can iterate.

[Bryce Harrington (bryce)]

Btw, I see upstream has a quirk fix for an RV530 macbook issue, so I'm going to go ahead and pull that in, and make a big assumption that it fixes the issue. If not, please go ahead and reopen this bug, and install the debs in my previous comment, and report back with the Xorg.0.log and we'll go from there.

[Jordan (jordanu)]

I guess the mirror I am using hasn't gotten your new packages yet ( I updated and still had the problem ) but installing the .debs you linked to has fixed the problem.

Thank you for your time and effort in getting this to work.

[Bryce Harrington (bryce)]

Jordan, wow that was lucky! Would you mind attaching your /var/log/Xorg.0.log, and I'll forward this patch upstream?

[Bryce Harrington (bryce)]

Jordan, would you mind testing one more thing:

http://www.bryceharrington.org/ubuntu/Ati/

That package includes the macbook fix from upstream that I mentioned. (My earlier upload didn't go through since we're in beta-freeze at the moment.) I want to determine whether that fixes the issue independently of the patch I made; if it does, there's no need for my patch, but if it doesn't, I should rework my patch a bit and use it.

Thanks again.

[Jordan (jordanu)]

The new packages bring the problem back so I guess that your patch is needed. I am attaching my Xorg.0.log.
Also, experimenting with radeonhd I noticed the same banding problems, should I file a new bug report against radeonhd?

[Oibaf (oibaf)]

Jordan, did you try the radeonhd packages from https://edge.launchpad.net/~tormodvolden/+archive (which are recent git snapshots)?

I don't have the banding problem in radeonhd with these updated packages.

[Bryce Harrington (bryce)]

Jordan, can you please post an Xorg.0.log from when the deb from http://bryceharrington.org/ubuntu/AtiDithering/ is installed?

What we're looking for is lines with "BWH:" in them in the log.

[Oibaf (oibaf)]

I noticed that I had the same problem with my MacBook Pro and your packages greatly diminishes the banding. Attached is the Xorg.0.log. The "BWH:" lines are:

...
Set CRTC 0 Source success
BWH: CHIP FAMILY is RV530
BWH: MonType MT_LCD
BWH: Not IS_DCE3_VARIANT
Output LVDS setup success
BWH: ATOM_DEVICE_LCD1_SUPPORT
Output 23 enable success
...

[Jordan (jordanu)]

BWH: CHIP FAMILY is RV530
BWH: MonType MT_LCD
BWH: Not IS_DCE3_VARIANT
BWH: ATOM_DEVICE_LCD1_SUPPORT

Also, I still get banding with the radeonhd package from https://edge.launchpad.net/~tormodvolden/+archive

[Bryce Harrington (bryce)]

Created an attachment (id=19363)
Adds dithering for rv530 lcd's

Two ubuntu users confirmed that enabling the dithering worked out in bug 17094 worked for the RV530. Attached is a patch to add support for this.

[Bryce Harrington (bryce)]

The Ubuntu bug is here:

https://bugs.edge.launchpad.net/ubuntu/+source/xserver-xorg-video-ati/+bug/264535

[Bryce Harrington (bryce)]

Thanks guys, it's nice to see you both have the same output - the same fix ought to solve it for both of you I think.

I've modified the earlier patch by limiting it to only turn dithering on for RV530 cards. It should solve it for you; if not please reopen this bug.

I've also forwarded the patch upstream; maybe they will have some suggestions to improve it.

[Launchpad Janitor (janitor)]

This bug was fixed in the package xserver-xorg-video-ati - 1:6.9.0+git20081003.f9826a56-0ubuntu2

---------------
xserver-xorg-video-ati (1:6.9.0+git20081003.f9826a56-0ubuntu2) intrepid; urgency=low

  * rv530_dithering.patch: Expands previous dithering fixup to the RV530
    (LP: #264535)

 -- Bryce Harrington <email address hidden> Fri, 03 Oct 2008 19:51:56 -0700

[Oibaf (oibaf)]

Was someone able to reproduce the problem with a RV530 on a machine other than a MacBook Pro?
What will happen when this fix is enabled on cards that does not need it?

[agd5f (agd5f)]

This should be fixed properly in commit:
a5c5ce96279d01eb519bfb92b94c06a58acb7f07

[Oibaf (oibaf)]

I tried current git (commit dc795ba66a8c0eb2c3fdb86bd42dfc17e0aa3b6e) using Ubuntu packages from https://edge.launchpad.net/~tormodvolden/+archive on an Ubuntu 8.04 (with xserver 1.4) but this is not yet fixed on my MacBook Pro. I can provide more informations if needed.

[agd5f (agd5f)]

Created an attachment (id=19490)
dump lvds misc info

(In reply to comment #3)
> I tried current git (commit dc795ba66a8c0eb2c3fdb86bd42dfc17e0aa3b6e) using
> Ubuntu packages from https://edge.launchpad.net/~tormodvolden/+archive on an
> Ubuntu 8.04 (with xserver 1.4) but this is not yet fixed on my MacBook Pro. I
> can provide more informations if needed.
>

I wouldn't put it past apple to have wrong or missing data for the LCD in their bios. Can you apply the attached patch and then attach your resulting log to this bug?

[Oibaf (oibaf)]

Created an attachment (id=19498)
Xorg.0.log with added debug line run on a MacBook Pro

Done. I get this:
LVDS misc: 0x69

Full log attached.

[Oibaf (oibaf)]

Note that I was one of the two were the Ubuntu patch worked.

[agd5f (agd5f)]

Created an attachment (id=19566)
possible fix

Looks like the grey level was being set wrong. This patch should fix it.

[Oibaf (oibaf)]

I am still having the "banding" with this patch.

[agd5f (agd5f)]

Created an attachment (id=19570)
possible fix

Can you try this patch?

[Oibaf (oibaf)]

Created an attachment (id=19571)
Xorg.0.log with latest pacth

Still same problem. Xorg log attached.

For this testing is sufficient to copy the just compiled:
xf86-video-ati/src/.libs/ati_drv.so
xf86-video-ati/src/.libs/radeon_drv.so

to /usr/lib/xorg/modules/drivers/ and reboot?

[agd5f (agd5f)]

Created an attachment (id=19578)
possible fix

Try this patch. I think probably just clearing the truncate bit should do it, but if that doesn't work uncomment this line:
//lvtma_bit_depth_control &= ~AVIVO_LVTMA_BIT_DEPTH_CONTROL_SPATIAL_DITHER_EN;
and try again. As for testing, all you need to do is copy the new files over and stop and start X:
sudo /etc/init.d/gdm stop
sudo /etc/init.d/gdm start

[Oibaf (oibaf)]

I tried the latest patch:
1) with the unmodified patch the banding is fixed, however there are some areas (where there is the colour transition) where the colour is not uniform (like if there were some random noise overlapped);
2) with the uncommented
//lvtma_bit_depth_control &= ~AVIVO_LVTMA_BIT_DEPTH_CONTROL_SPATIAL_DITHER_EN;
the problem is also greatly fixed (I can see very little visible banding looking closer, but I think this is how the image is really) without the noise overlap. Trying to look the same image on an intel graphic card I get the same behaviour.

I think the second one (with the uncommented line) should be preferred as it is similar to the intel card and also looks subjectively better.

Thanks for all your work on fixing this problem.

[Oibaf (oibaf)]

After 2 days of testing I would say that the behaviour with the commented
//lvtma_bit_depth_control &= ~AVIVO_LVTMA_BIT_DEPTH_CONTROL_SPATIAL_DITHER_EN;
is buggy and that that line should be uncommented.

[agd5f (agd5f)]

Created an attachment (id=19664)
final fix

Ok, this is what I plan to commit. Please verify that it works.

[Oibaf (oibaf)]

I confirm it's working as expected.

Thanks,
Fabio

[agd5f (agd5f)]

thanks! committed: 435cf7da68186f2601c4b888296117d4f652c625

[Tormod Volden (tormodvolden)]

Packages with the new upstream fix for MacBook (435cf7da68186f2601c4b888296117d4f652c625) in my PPA.

[Oibaf (oibaf)]

See also bug #284346

[cazacugmihai (cazacugmihai)]

Hi,

I have the same problem. Please read: https://bugs.launchpad.net/ubuntu/+source/xserver-xorg-video-intel/+bug/305909

Thanks.

[cazacugmihai (cazacugmihai)]

Attached is a screenshot.