Only half of RAM useable when using Device Tree on Panda board

[Tom Gall]

the panda board has 1 Gig of RAM.

Only half of it is being mapped by the kernel.

top - 17:50:26 up 1:33, 2 users, load average: 0.00, 0.01, 0.29
Tasks: 71 total, 1 running, 70 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 459704k total, 389956k used, 69748k free, 46128k buffers
Swap: 0k total, 0k used, 0k free, 303232k cached

root@localhost:~# cat /proc/meminfo
MemTotal: 459704 kB
MemFree: 69780 kB
Buffers: 46128 kB

...

I've heard there might be issues involving high mem however I have not personally looked into that. For natty this should be addressed. We should be able to utilize 768M without running afoul of highmem shouldn't we?

[Steve Langasek (vorlon)]

RAM is missing because we're configuring the system to be brought up that way with linaro-media-create when it sets the default boot args; I haven't seen any confirmation that bug #633227, which this bug was marked as a duplicate of, has been reproduced with the Linaro kernel (that bug is filed against the Ubuntu omap4 kernel build, not against the Linaro kernel). On the other hand, I have gotten feedback from Marcin that we're being too conservative in mapping memory on panda, deliberately mapping < 512MB of RAM; see the linked branch for a proposed fix.

If the panda is stable with this fix applied, I think that's the solution to this bug.

[Loïc Minier (lool)]

I just tweaked this slightly after reading comments in bug #633227 where it said it should be a multiple of 8 MiB; so the first length is now 456 MiB instead of 463 MiB.

[Marcin Juszkiewicz (hrw)]

[ 0.000000] Memory: 463MB 480MB = 943MB total
[ 0.000000] Memory: 934628k/934628k available, 63772k reserved, 229376K highmem
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
[ 0.000000] fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)
[ 0.000000] DMA : 0xffc00000 - 0xffe00000 ( 2 MB)
[ 0.000000] vmalloc : 0xf0800000 - 0xf8000000 ( 120 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xf0000000 ( 768 MB)
[ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
[ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
[ 0.000000] .init : 0xc0008000 - 0xc004e000 ( 280 kB)
[ 0.000000] .text : 0xc004e000 - 0xc07dcd34 (7740 kB)
[ 0.000000] .data : 0xc07de000 - 0xc0843488 ( 406 kB)
hrw@panda:~$ free -m
             total used free shared buffers cached
Mem: 921 875 45 0 503 124
-/+ buffers/cache: 247 673
Swap: 0 0 0
hrw@panda:~$

Good to know that my Pandaboard does not read Launchpad bug reports - 921 is not multiple of 8 and works fine.

[Marcin Juszkiewicz (hrw)]

root@linaro:~# uname -a
Linux linaro 2.6.38-1002-linaro-omap #3-Ubuntu SMP Fri Apr 15 14:00:54 UTC 2011 armv7l armv7l armv7l GNU/Linux
root@linaro:~# cat /proc/cmdline
console=tty0 console=ttyO2,115200n8 root=UUID=57bd55a1-fbe2-45be-ab2b-ea753174fa46 rootwait ro earlyprintk fixrtc nocompcache vram=32M omapfb.vram=0:8M mem=456M@0x80000000 mem=512M@0xA0000000
root@linaro:~# free -m
             total used free shared buffers cached
Mem: 662 249 412 0 12 113
-/+ buffers/cache: 123 539
Swap: 0 0 0

dmesg:

[ 0.000000] Memory: 456MB 220MB 260MB = 936MB total
[ 0.000000] Memory: 674048k/674048k available, 55040k reserved, 0K highmem

Thats with latest hwpack. So we still miss memory in official Linaro kernel (Ubuntu kernel gives memory correctly).

[Mattias Backman (mabac)]

Still in nano image from 2011-05-05 with hwpack from same date:

root@linaro:~# uname -a
Linux linaro 2.6.38-1002-linaro-omap #3-Ubuntu SMP Fri Apr 15 14:00:54 UTC 2011 armv7l armv7l armv7l GNU/Linux
root@linaro:~# cat /proc/cmdline
console=tty0 console=ttyO2,115200n8 root=UUID=bb58cb88-f4b2-4dc8-af15-fcb090d7c7b4 rootwait ro earlyprintk fixrtc nocompcache vram=32M omapfb.vram=0:8M mem=456M@0x80000000 mem=512M@0xA0000000
root@linaro:~# free -m
             total used free shared buffers cached
Mem: 662 98 564 0 1 80
-/+ buffers/cache: 16 646
Swap: 0 0 0
root@linaro:~# dmesg | grep Memory
[ 0.000000] Memory policy: ECC disabled, Data cache writealloc
[ 0.000000] Memory: 456MB 220MB 260MB = 936MB total
[ 0.000000] Memory: 674316k/674316k available, 54772k reserved, 0K highmem

[Tom Gall (tom-gall)]

still present on 0519 hwpack using the developer image dated the same.

[ 0.000000] Memory policy: ECC disabled, Data cache writealloc
[ 0.000000] Memory: 456MB 220MB 260MB = 936MB total
[ 0.000000] Memory: 674316k/674316k available, 54772k reserved, 0K highmem

[Marcin Juszkiewicz (hrw)]

same with linux-image-2.6.38-1003-linaro-omap_2.6.38-1003.4~ppa2_armel.deb kernel

[Dave Martin (dave-martin-arm)]

If we don't feel happy turning CONFIG_HIGHMEM on, could we use CONFIG_VMSPLIT_2G instead? This allows Linux to map and use all of the physical RAM, though it does reduce the virtual address space available to user processes to 2G.

This gives me something more like:
[ 0.000000] Memory: 472MB 480MB = 952MB total
[ 0.000000] Memory: 950428k/950428k available, 57188k reserved, 0K highmem

(note that this isn't exactly the current Linaro kernel config; with CONFIG_VMSPLIT_3G I had about 7xxxxxk of memory available to Linux)

[Ricardo Salveti (rsalveti)]

We should be able to use HIGHMEM just fine with this kernel, we just need to change the kernel config file. Ubuntu has 1GB support with HIGHMEM for quite some time already, and it works fine.

[John Rigby (jcrigby)]

I will turn on HIGHMEM for the next build but only for omap. I will turn it on for all platforms for oneiric.

[Ricardo Salveti (rsalveti)]

On Fri, May 20, 2011 at 12:24 PM, John Rigby <email address hidden> wrote:
> I will turn on HIGHMEM for the next build but only for omap.  I will
> turn it on for all platforms for oneiric.

Fair enough.

[Ricardo Salveti (rsalveti)]

Kernel support is there already, but still unable to use 1GB.

It seems the issue is related with the device tree support, as once I avoid loading the board.dtb file at u-boot I'm able to see and use all memory. With the board.dtb file loaded most memory will be at the reserved area.

[Grant Likely (glikely)]

Without looking at both DT and non-DT boot logs, I suspect the problem is that U-Boot is only reporting 512MB of ram, and stuffing that amount into the /memory node in the DT. It looks like in the non-DT use case, the kernel command line is being used to manually set the RAM layout, probably so that the FB region doesn't get used by the kernel. In the DT, a couple of memreserve blocks are used to the same purpose:

/memreserve/ 0x9D000000 0x03000000; /* Frame buffer */
/memreserve/ 0xB0000000 0x10000000; /* Top 256MB is unaccessable */

I put those memreserve lines in, but I've never done the due diligence to ensure they are actually accurate.

[David Long (dave-long)]

With DT present I see the regions described in the .dtb file being properly excluded. So now the problem is that just under 3/4 of available RAM is visible, when we should be able to see all except the 48MB set aside just below the 1/2G boundary (which is used by other processor(s) on the SoC).

The comment in the .dts file is not quite right. The top 256MB *is* (or can be) accessible with HIGHMEM condfigured. However, when DT is present the dtb data (and the initrd) are copied to the very end of SDRAM by uboot, and a pointer to that device tree location is passed into the kernel startup code. Apparently the kernel cannot handle device tree (and maybe initrd data too) residing in HIGHMEM.

[Grant Likely (glikely)]

Hmmm, I had hoped that would not be a limitation with Rob Herrings patch to allow DTB/ATAGs to be mapped during early setup. I guess we need to force u-boot to keep things lower. I believe the easy fix is to define CONFIG_SYS_BOOTMAPSZ to be something like 512MB in u-boot.

[Riku Voipio (riku-voipio)]

Still an issue with 11.06 release candidate, testing with pandaboard A1 and developer image.

Linux version 2.6.38-1003-linaro-omap (buildd@adoxaceae) (gcc version 4.5.2 (Ubuntu/Linaro 4.5.2-8ubuntu4) ) #4~ppa5-Ubuntu SMP PREEMPT Wed May 25 14:44:32 UTC 2011 (Ubuntu 2.6.38-1003.4~ppa5-linaro-omap 2.6.38.7)

[David Long (dave-long)]

A fix for this is currently being reviewed. It involves changes to both u-boot and linaro-media-create. When in place we can remove the disabling of the last 1/4GB from our Panda .dts file.

[David Long (dave-long)]

The u-boot fdt_highmem change has been taken into the upstream pool.

After some discussion it seems agreed an additional change is desired to also allow u-boot to auto-relocate fdt and initrd data to a new (lowmem) location for booting. That change will be independent of this one.

[Chase Qi (chase-qi)]

Migrated to bugzilla: https://bugs.linaro.org/show_bug.cgi?id=550