noexec doesn't apply on 32-bit AMD64

[John Moser]

Binary package hint: linux-source-2.6.19

The Linux kernel is supposed to enforce PROT_EXEC properly by default on any system with a hardware NX bit. The Athlon 64 has such a hardware bit; but it is not enforcing it.

I'm running on an Athlon 64 in 32-bit mode, running 32-bit Ubuntu with kernel 2.6.19 (mostly Feisty). Apparently, 'noexec=on' on the kernel command line does nothing; the NX bit seems to not work. This is contradictory with the kernel Documentation.

Chunk of my /proc/cpuinfo showing NX capability:

flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext lm 3dnowext 3dnow up ts fid vid ttp

I will attach a test program that attempts to disable PROT_EXEC for a page of memory containing (I believe) the entry point of a function. It's compiled as such:

$ gcc -O2 -shared -fpic test_so.c -o test_so.so
$ gcc -O2 test.c -o test -ldl

Running it on AMD64-ubuntu gives the following output:

$ ./test
Test function run successfully!
Segmentation fault

This is good; I tried to execute non-executable memory, it segfaulted. However, 32-bit Ubuntu on the Athlon64 gives the following:

$ ./test
Test function run successfully!
Test function run successfully!

Apparently noexec is not being honored.

This seems to affect Dapper as well with 2.6.15; edgy is probably also affected, but I Haven't tested 2.6.17.

[John Moser (nigelenki)]

This source file generates a shared object for the test program. This shared object is abused by the test program; it is protected non-executable before the second call is made.

Compiled by:

$ gcc -O2 -shared -fpic test_so.c -o test_so.so

[John Moser (nigelenki)]

This source file generates a test program. This test program calls a function in test_so.so twice; before the second time, it marks the page containing the function entry point as PROT_READ|PROT_WRITE.

This test program should segfault when it attempts to execute the function the second time; the memory it attempts to execute will be non-executable during the second call.

Compiled by:

$ gcc -O2 test.c -o test -ldl

test_so.so must be in the current working directory.

[Kyle McMartin (kyle)]

We're not turning on HIGHMEM64G on desktop kernels.

[John Moser (nigelenki)]

A better explanation would be.. well.. better? I suppose I'll read this as, "We're not turning on basic security features on desktop kernels."

[Kyle McMartin (kyle)]

PAE isn't supported by some processors... Keep in mind it was added with the Pentium Pro, so i586 machines won't boot. Not only that, Intel released a model of the Pentium M *recently* which doesn't support PAE.

[John Moser (nigelenki)]

Thanks, that's a more interesting answer. I'll look into if there's a way to detect PAE mode availability, and see if I can come up with a better solution than switching on 64G memory support.

[John Moser (nigelenki)]

I am going to look into a useful way to have PAE and non-PAE in the same kernel (no function pointers, and alternatives would be a mess too). Both sets of code would have to be present... macros... hrm.

[LKRaider (paul-eipper)]

Any update on this? Is it really possible to make it work selectively on certain platforms?

[Yuhong Bao (yuhongbao-386)]

"Not only that, Intel released a model of the Pentium M *recently* which doesn't support PAE."
Yep, pre-2005 Pentium Ms and Celeron Ms that didn't support NX didn't support PAE either. And don't forget Transmeta, who did not add PAE and NX support until the Efficeon, or VIA, who did not add PAE and NX support until the C7. And AMD's Geode does not support PAE or NX. Also older versions of VMware and even current versions of Virtual PC, Virtual Server, and Parallels did not support PAE or NX. And some versions of VMware that did support PAE required that the line "paevm=true" to enable the support. Newer versions of VMware, Hyper-V, and Parallels Server does support PAE and NX, however.

[Yuhong Bao (yuhongbao-386)]

At least on the guest. On the host, Parallels Workstation 2.0 at first did not support PAE on the host at all, both on Windows and on Linux (in which you get a error during compile of the kernel modules if you try to do so against a PAE-enabled kernel). Parallels Workstation 2.1 did support PAE, but only if the physical address space is limited to 32-bit, and each OS have a different way of during this. NX can still be used, however.

[Magnus S (magnuss)]

Closing this bug since it has been expired a long time ago.