linux-image-2.6.28-5-powerpc has ioremap fault with radeon drm module

[Jonathan Hudson]

Binary package hint: linux-image-powerpc

After a Jaunty sync of c. 18:00 UTC 2009-03-19, the new linux-image-2.6.28-5-powerpc kernel has a problem with the kernel drm module, resulting in:

* No 3D video acceleration;

* Kernel log messages like:
Mar 19 18:29:41 tigger kernel: [ 50.696676] agpgart-uninorth 0000:00:0b.0: putting AGP V2 device into 4x mode
Mar 19 18:29:41 tigger kernel: [ 50.696694] radeonfb 0000:00:10.0: putting AGP V2 device into 4x mode
Mar 19 18:29:42 tigger kernel: [ 51.052735] __ioremap(): phys addr 0x0 is RAM lr e2da9d5c
Mar 19 18:29:42 tigger kernel: [ 51.052748] __ioremap(): phys addr 0x101000 is RAM lr e2da9d5c
Mar 19 18:29:42 tigger kernel: [ 51.052754] __ioremap(): phys addr 0x102000 is RAM lr e2da9d5c
Mar 19 18:29:42 tigger kernel: [ 51.052762] [drm:radeon_do_init_cp] *ERROR* could not find ioremap agp regions!

Rebooting into 2.6.28-4-powerpc (or pretty much any previous kernel) results in a working system again, no log messages and working 3D. This is newly introduced in 2.6.28-5-powerpc.

-jh

[Bugzi09-fdo-tormod (bugzi09-fdo-tormod)]

Created an attachment (id=25076)
dmesg with drm debug=1

Forwarded from Ubuntu bug https://bugs.launchpad.net/bugs/345542

DRM fails to initialize on PowerPC with Ubuntu kernel 2.6.28-5-powerpc, based on 2.6.28.1 (the 2.6.28-4-powerpc, based on 2.6.28, worked fine).

[drm:radeon_do_init_cp] *ERROR* could not find ioremap agp regions!

Unlike the problem with 2.6.29 reported earlier (http://bugzilla.kernel.org/show_bug.cgi?id=12441), "nopat" does not help.

The original reporters will also try newer kernels or build newer drm modules, but the reduced support for PowerPC in Ubuntu makes it more challenging.

[Tormod Volden (tormodvolden)]

Can you please try to boot with the "nopat" boot parameter?

[Jonathan Hudson (jh+lpd)]

$ uname -a
Linux tigger 2.6.28-6-powerpc #20-Ubuntu Fri Apr 17 08:30:40 UTC 2009 ppc GNU/Linux

$ cat /proc/cmdline
root=/dev/hda3 ro nopat quiet splash

[ 53.395940] agpgart-uninorth 0000:00:0b.0: putting AGP V2 device into 4x mode
[ 53.395959] radeonfb 0000:00:10.0: putting AGP V2 device into 4x mode
[ 53.760940] __ioremap(): phys addr 0x0 is RAM lr e2f2fd10
[ 53.760953] __ioremap(): phys addr 0x101000 is RAM lr e2f2fd10
[ 53.760959] __ioremap(): phys addr 0x102000 is RAM lr e2f2fd10
[ 53.760968] [drm:radeon_do_init_cp] *ERROR* could not find ioremap agp regions!

I think that means it does not help.

-jonathan

[Sergey V. Udaltsov (sergey-udaltsov)]

Same here. nopat makes no difference

[Tormod Volden (tormodvolden)]

Thanks. Yes, unfortunately it was not that simple. Is 2.6.28-4-powerpc the latest kernel on powerpc?

In lack of newer kernels, I can suggest building updated drm modules separately. There is a drm-modules-sources package at https://launchpad.net/~xorg-edgers/+archive/ppa (in drm-snapshot) which after installation can build and install the modules with "sudo module-assistant auto-install drm-modules".

[Tormod Volden (tormodvolden)]

Please also provide dmesg after loading the drm module with its debug=1 option:
1. Shut down X and "modprobe -r drm" (or boot with "text" boot parameter).
2. "modprobe drm debug=1"
3. Start X (for instance with "startx")

[Tormod Volden (tormodvolden)]

"modprobe -r radeon drm" it should probably be in (1) above.

[Sergey V. Udaltsov (sergey-udaltsov)]

The dmesg after setting debug=1

[Michel Dänzer (michel-daenzer)]

I haven't seen this on my PowerBook with upstream kernels up to 2.6.29.1, so 'on powerpc' may be too broad.

One thing I notice is that the dmesg seems to be from a 64 bit kernel. Are people also seeing this on machines running 32 bit kernels?

> DRM fails to initialize on PowerPC with Ubuntu kernel 2.6.28-5-powerpc, based
> on 2.6.28.1 (the 2.6.28-4-powerpc, based on 2.6.28, worked fine).

So it should be possible to isolate the kernel change that causes or at least triggers it?

> Unlike the problem with 2.6.29 reported earlier
> (http://bugzilla.kernel.org/show_bug.cgi?id=12441), "nopat" does not help.

Yeah, PAT is an x86 feature...

[Sergey V. Udaltsov (svu)]

2.6.28-6-powerpc64-smp

I do not have 32-bit kernel.

[Bugzi09-fdo-tormod (bugzi09-fdo-tormod)]

> I haven't seen this on my PowerBook with upstream kernels up to 2.6.29.1, so
> 'on powerpc' may be too broad.

I only added it to the title to narrow it down to powerpc (and to differentiate it from the i386 issue with the same error message), I didn't mean to imply it touches all powerpc. Please correct the title if it can be understood the wrong way.

> So it should be possible to isolate the kernel change that causes or at least
> triggers it?

I don't know if we can hope that anyone will bisect it, so I was hoping for an educated guess from someone here :) At least I checked the Ubuntu changes between those two versions and I could not see anything relevant AFAICS. There are deb's for vanilla kernels, but unfortunately not built for powerpc. I hope someone will at least built the newer drm modules and check.

[Bugzi09-fdo-tormod (bugzi09-fdo-tormod)]

Two other guys are using 32 bit kernel (on G4's).

[Tormod Volden (tormodvolden)]

Question from upstream: Is this a 64-bit kernel? Does it happen with a 32-bit kernel?

[Sergey V. Udaltsov (sergey-udaltsov)]

2.6.28-6-powerpc64-smp

[Jonathan Hudson (jh+lpd)]

32-bit kernel on an old G4 1Ghz.

[David Hedberg (david-hedberg)]

As one of those affected, I'm using a 32 bit kernel (old ibook G4).

[Tormod Volden (tormodvolden)]

I noticed there are both 32 and 64 bit kernels on the live CD at http://cdimage.ubuntu.com/ports/daily-live/current/ , but I don't know more about it.

[Andy Whitcroft (apw)]

Reassigning to the ports kernel package as this is a real kernel issue rather than an issue with the -meta package.

[Jeremy Visser (jeremy-visser)]

I'm affected by this issue (1GHz eMac) as well and can help test things.

In the meantime, is there a version of that old kernel floating around that I can downgrade to so I can get my 3D working again? My Mac is virtually unusable without it.

[Jonathan Hudson (jh+lpd)]

As per description of initial bug, 2.6.28-4-powerpc is the last working kernel (which I still have in /var/cache/apt/archives if you can't find it anywhere else ...)

-jh

[Sergey V. Udaltsov (sergey-udaltsov)]

Is there a way to analize the difference between the latest kernel ans 2.6.28-4 - to find the reason of the broken ioremap?

[Tormod Volden (tormodvolden)]

You can look through the Ubuntu changelog and see if any Ubuntu changes were made, then (since the -4 is based on 2.6.28 and the -5 on 2.6.28.1) check the upstream changelog between 2.6.28 and 2.6.28.1.

If someone can build a vanilla 2.6.28 and 2.6.28.1 and test and compare, to rule out any Ubuntu changes, it would be great.

[Bugzi09-fdo-tormod (bugzi09-fdo-tormod)]

According to http://lists.x.org/archives/xorg-devel/2009-May/000825.html this is a bug in the Ubuntu kernel.

[Tormod Volden (tormodvolden)]

According to http://lists.x.org/archives/xorg-devel/2009-May/000825.html this is a bug in the Ubuntu kernel. Vanilla kernel should work fine.

[spydouglas (ben-wibking)]

Should this be tagged as regression-release, given that it now affects jaunty as released?

[Tormod Volden (tormodvolden)]

In principle yes, but is Jaunty really "released" for powerpc? I hope there will be some newer powerpc kernels.

[Tormod Volden (tormodvolden)]

Can you please also add /var/log/dmesg from a working and a non-working kernel?

[David Hedberg (david-hedberg)]

I don't have anything new to come with, but just so that it doesn't get lost:

As I stated in Bug #354677 (a duplicate of this one), a temporary work around that works for me is to add

  Option "BusType" "PCI"

to the Device section in /etc/X11/xorg.conf . I am assuming this comes with a slight performance penalty, but at least it works.

[Jonathan Hudson (jh+lpd)]

Dmesg file 2.6.28-6-powerpc for failure case

[Jonathan Hudson (jh+lpd)]

Dmesg file2.6.28-4-powerpc for non-failure case

[Sergey V. Udaltsov (sergey-udaltsov)]

Confirming the workaround. BusType PCI makes things way better.

[Simon Butcher (sbutcher)]

can you attach the entire xorg.conf? mine is empty (0 bytes)

[Tormod Volden (tormodvolden)]

There is now a 2.6.30 kernel for powerpc, please check if the issue has been fixed: https://launchpad.net/ubuntu/+source/linux-ports/

[Sergey V. Udaltsov (sergey-udaltsov)]

Simon, as asked, here is my xorg.conf

Tormod, is 2.6.30 safe for jaunty? It is in the karmic section...

[David Hedberg (david-hedberg)]

I installed the new kernel from linux-image-2.6.30-1-powerpc_2.6.30-1.1_powerpc.deb on my ibook G4, but it greets me with a kernel BUG/Oops and a panic early during the boot process so I can't really try it out.

[Jonathan Hudson (jh+lpd)]

Same here. Here's a image of the panic dump if that helps anyone (note that the green dots are normal on my old machine in console mode).

[Tormod Volden (tormodvolden)]

> Tormod, is 2.6.30 safe for jaunty? It is in the karmic section...

At least for i386 it is not a problem to use the Karmic kernel in Jaunty, generally any newer kernel works fine. Also, since it installs next to (and does not replace) the old kernel, there is no risk, you can always choose the old kernel when booting.

> it greets me with a kernel BUG/Oops and a panic early during the boot process

Seems like the oops happens before any userspace is detected, so it is not a Jaunty-Karmic compatibility issue. Please report it against the linux-ports package.

[Jonathan Hudson (jh+lpd)]

https://bugs.launchpad.net/bugs/373131

[Bugzi09-fdo-tormod (bugzi09-fdo-tormod)]

I think Alex was onto something in that thread: http://lists.x.org/archives/xorg-devel/2009-May/000816.html

The Ubuntu changes introduced in this version is mainly his jumbo patch "radeon: add support for RS600, R6xx, and R7xx GPUs" http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-jaunty.git;a=commit;h=85568da345d4348f33c69043c9bc4e78ad819706

[spydouglas (ben-wibking)]

Per a previous comment, I compiled a vanilla kernel (2.6.28.10) and DRI works fine. I can also compile and test an earlier 2.6.28 release if that would help isolate the issue.

[Tormod Volden (tormodvolden)]

spydouglas, thanks, that is very useful. I understand I was wrong about the upstream versions, since the "linux-ports" versions do not match the "linux" versions: linux-ports 2.6.28-5.11 is based on linux 2.6.28-11.34 which again is based on upstream 2.6.28.8, whereas the linux-ports 2.6.28-4.10 is based on linux 2.6.28-9.29 (upstream 2.6.28.7)

So I guess testing the vanilla 2.6.28.8 makes most sense. If it works, the issue is caused by the Ubuntu patches on top. In particular, I see http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-jaunty.git;a=commit;h=85568da345d4348f33c69043c9bc4e78ad819706 was added. Did anyone try building drm modules? Either on the failing or the last good kernel.

[Tormod Volden (tormodvolden)]

The author of the above patch commented on IRC:
 - maybe some issue with ioremap_wc() on the ubuntu kernel
 - upstream kernels use ioremap_wc() already
 - drm in upstream kernels

[Jonathan Hudson (jh+lpd)]

Shortly after all this started, I built linux-source_2.6.28.11.15_all.deb on PPC to see if that later version (the source for then current ia32 version) had the fault, and it did on PPC. Yesterday, following spydouglas' example, I built kernel.org 2.6.28.10, which, as we now know, works on PPC.

This morning, inspecting the differences in drivers/gpu/drm/radeon/radeon_cp.c, I naively changed the three instances of drm_core_ioremap_wc() in the above Ubuntu source (drivers/gpu/drm/radeon/radeon_cp.c) to remove the _wc, i.e. back to the Korg non write combining version (this is at lines 1183-1185), [ ==> drm_core_ioremap()].

The kernel from modified Ubuntu source works with DRM, though I have no idea what the side affects might be.

-jh

[spydouglas (ben-wibking)]

Further confirmation probably isn't necessary, but I've compiled and tested 2.6.28.8 with success. Also, I've been having stability issues with X since I've used the BusMode workaround, but I'm not sure yet if they're related.

[Luke Yelavich (themuso)]

Thanks for this find. It should be easier to track down the cause of this bug with git annotate on the powerpc kernel source tree for Ubuntu.

I also have an iBook G4 that appears to be showing the same behavior, so testing the eventual fix should be easy for me at least.

 affects ubuntu/linux-ports
 status confirmed
 assignee themuso

[Jonathan Hudson (jh+lpd)]

I built K.org 2.6.29.2 and that works with DRM; this is probably significant in that it uses the write combining drm_core_ioremap_wc(), so it would appear that the Ubuntu patches for write combining are the culprit:

In summary:

Ubuntu-pcc 2.6.28-6-powerpc [drm_core_ioremap_wc()] - fails
Kernel.org 2.6.28.10 [drm_core_ioremap()] - works
Ubuntu 2.6.28.9 (*2.6.28.11.15_all.deb) [drm_core_ioremap_wc()] - fails
Ubuntu 2.6.28.9 hacked, remove _wc (*2.6.28.11.15_all.deb) [drm_core_ioremap()] - works
Kernel.org 2.6.29.2 [drm_core_ioremap_wc()] - works

-jh

[Luke Yelavich (themuso)]

Ok, testing on my iBook, the changes in function calls that jh suggested work. I shall get this fixed up in a kernel update, along with a few other bits and pieces.

[vanhammersly (michnaugh1)]

I had the same issue. I installed 2.6.29.4 and it resolved the issue. Here's a link to compiling and installing a kernel in powerpc jaunty.

http://ubuntuforums.org/showthread.php?t=453497

--Mike

[Randall Ross (randall)]

Are there plans to incorporate this working kernel into an update?

--
Contact me this way:
<email address hidden>
sms: [phone]
xmpp://<email address hidden>
http://identi.ca/rrnwexec/

[Reimundo Heluani (heluani)]

runnung vanilla 2.6.29.4 on PPC Powermac G5 (ATI Radeon 9600 RV350)
glxinfo shows Direct rendering:yes, but still using the software rasterizer

[S. D. (akjag)]

Also confirmed, 1.67GHz G4 Ti-Book, using jaunty's PPC kernel (v. 2.6.28-6-powerpc). Blue-background & all.

Found bug #370200 as well... relevant or no??

Has Luke really been working on a kernel update for the past two months? I hope it's ready now :-)

[Luke Yelavich (themuso)]

Unfortunately due to powerpc not being an officially supported architecture, I don't think the updates I wanted to make to this kernel qualify for a stable release update. I will however build a new set of kernels and make them available for people to use, with this fix included.

 affects ubuntu/linux-ports
 status wontfix
 assignee nobody