-mlong32 is an unknown option

[indigoviolet]

Binary package hint: gcc-3.3

I'm only transferring information from this (Donald Knuth's) page http://www-cs-faculty.stanford.edu/~uno/news.html, since I'm guessing Knuth did not report this himself.

A Flame About 64-bit Pointers

It is absolutely idiotic to have 64-bit pointers when I compile a program that uses less than 4 gigabytes of RAM. When such pointer values appear inside a struct, they not only waste half the memory, they effectively throw away half of the cache.

The gcc manpage advertises an option "-mlong32" that presumably does what I want. Namely, it should compile code for my x86-64 architecture, taking advantage of the extra registers etc., but it should also know that my program is going to live inside a 32-bit virtual address space.

Unfortunately, the gcc I got with Ubuntu 7.10 says that -mlong32 is an unknown option. Probably that happens because programs compiled with this convention will need to be loaded with a special version of libc.

Please, somebody, make that possible.

[Tollef Fog Heen (tfheen)]

* indigoviolet

| Unfortunately, the gcc I got with Ubuntu 7.10 says that -mlong32 is an
| unknown option. Probably that happens because programs compiled with
| this convention will need to be loaded with a special version of libc.

Not only would you need a different libc, you would have to have _all_
other libraries compiled with this same other ABI. I suspect you
would have to get support into the kernel as well for it.

If you really want to do this, it would probably be better to approach
AMD and Intel with the suggestion of defining such an ABI; this is not
something Ubuntu can reasonably do (and I'll therefore mark this bug
as «Won't Fix»).

--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

[indigoviolet (f-launchpad-indigoviolet-spamgourmet-com)]

Oh well. Thanks. I only wanted to bring it to your attention. Atleast you now know that Knuth uses Ubuntu 7.10 :)

[indigoviolet (f-launchpad-indigoviolet-spamgourmet-com)]

I received this today, cc-ed to Tollef.

----------------
Dear Tollef,

Regarding "Ubuntu bug 185263":

Apparently it has never occurred to you that this is a bug
in the MAN file for gcc?

I admire the goals of Ubuntu. But how will I ever be able to
recommend it to friends and colleagues, if its implementors
have such callous regard for the quality of user documentation?

Sincerely, Don Knuth

[Jeff Bailey (jbailey)]

Please stop third-party relaying messages from people. All that's happening is that we're reply to bug reports that the actual reporter may or may not be seeing.

The man pages are *not* buggy. I refer everyone to the relevent section:

           MIPS Options -EL -EB -march=arch -mtune=arch -mips1 -mips2
           -mips3 -mips4 -mips32 -mips32r2 -mips64 -mips16 -mno-mips16
           -mabi=abi -mabicalls -mno-abicalls -mshared -mno-shared -mxgot
           -mno-xgot -mgp32 -mgp64 -mfp32 -mfp64 -mhard-float
           -msoft-float -msingle-float -mdouble-float -mdsp -mpaired-single
           -mips3d -mlong64 -mlong32 -msym32 -mno-sym32 -Gnum -membed‐
           ded-data -mno-embedded-data -muninit-const-in-rodata
           -mno-uninit-const-in-rodata -msplit-addresses -mno-split-addresses
           -mexplicit-relocs -mno-explicit-relocs -mcheck-zero-division
           -mno-check-zero-division -mdivide-traps -mdivide-breaks -mmemcpy
           -mno-memcpy -mlong-calls -mno-long-calls -mmad -mno-mad
           -mfused-madd -mno-fused-madd -nocpp -mfix-r4000 -mno-fix-r4000
           -mfix-r4400 -mno-fix-r4400 -mfix-vr4120 -mno-fix-vr4120
           -mfix-vr4130 -mfix-sb1 -mno-fix-sb1 -mflush-func=func
           -mno-flush-func -mbranch-likely -mno-branch-likely -mfp-exceptions
           -mno-fp-exceptions -mvr4130-align -mno-vr4130-align

Even the full documentation section is under "MIPS Options".

While this would probably be a fascinating option for amd64 to grow (ia64 has the option for an ILP32 mode on HP-UX, for instance), I suspect that the likelyhood of desktop systems not having >4GB of Ram in the near future is pretty slim (and certainly will be true by the time toolchain, kernel, and system libraries would have the capability). For servers, having more than 4GB is already reasonable and common.

[indigoviolet (f-launchpad-indigoviolet-spamgourmet-com)]

Jeff,

Under normal circumstances, I would agree completely. However, I do think exceptions can be made for persons of the stature of Don Knuth (look him up if that doesn't ring a bell). The reporter in this case _has_ seen the response, which is why his response was posted back here. Why does it matter who does the actual posting?

I think his point about the man page bug was that if the -mlong32 option is not recognized on amd64, it should not be in the man page. As you can see from your post, it in fact is. This is misleading.

[Jeff Bailey (jbailey)]

The manpage isn't misleading in the sense that it's wrong in any way. The manpage is misleading in that manpages such if they take more than a page or two of screen real-estate.

The problem with him not reporting this bug is that you and I are now stuck having this conversation, rather than the more reasonable case of the person having the problem with me or one of the other toolchain folks. I can't turn around to you and say "Tell me more about your problem so that I can help come up with an appropriate answer within the constraints of our resources" and get a meaningful answer. The fact that Donald is a reporter who could actually meaningfully give input to the question makes it that much more frustrating. =/

Marking as invalid. Documentation and compiler are consistent and correct - this option does not exist for amd64.

[Adam Buchbinder (adam-buchbinder)]

If it's an upstream problem (as Tollef said, "Not only would you need a different libc, you would have to have _all_ other libraries compiled with this same other ABI. I suspect you would have to get support into the kernel as well for it."), where's the best place to start filing wishlist bugs for it? The gcc bugzilla? The glibc bugzilla? The kernel bugzilla? Is this entirely impossible without hardware support from Intel and AMD? (Given that the request is simply for the option to compile some applications to use 32-bit pointers and be restricted to 4GB of address space, perhaps the problem can be solved at the virtual memory level rather than requiring specialized hardware?)

[Jeff Bailey (jbailey)]

On Mon, May 5, 2008 at 6:46 AM, Adam Buchbinder
<email address hidden> wrote:
> If it's an upstream problem (as Tollef said, "Not only would you need a
>
> different libc, you would have to have _all_ other libraries compiled
> with this same other ABI. I suspect you would have to get support into
> the kernel as well for it."), where's the best place to start filing
> wishlist bugs for it? The gcc bugzilla? The glibc bugzilla? The kernel
> bugzilla? Is this entirely impossible without hardware support from
> Intel and AMD? (Given that the request is simply for the option to
> compile some applications to use 32-bit pointers and be restricted to
> 4GB of address space, perhaps the problem can be solved at the virtual
> memory level rather than requiring specialized hardware?)

Well, let's think this through for a sec. The goal is to have 32-bit
pointers to better use processor cache, but still use the enhanced
instructions and registers provided by the amd64 instructions. This
is off the top of my head without doing any research at all. That
means I've likely missed at least one step in here.

This is probably possible on the hardware, although I haven't looked
at it directly. Assuming it is:

You first need to define an ABI document describing what registers and
such would be used for calling conventions.

You then need to talk to the toolchain folks to get a GNU triple
assigned to this mode.

The kernel would have to be hacked to setup the address space
appropriately (make sure that the vDSO and such were located in low
enough memory).

Binutils would need hacking to understand that triple, and maybe some
tweaks to the assembler to check for invalid pointer sizes, and some
tweaks to the linker.

gcc would need hacking to restrict itself to a 32bit address space,
while teaching it about the additional registers and instructions.

Then you'd need to hack on glibc to make sure that it knew the triple
and would build. This would also mean hacking any custom asm files so
that they constrained themselves appropriately. There would probably
be some elf changes as well to match the ABI document.

After all this had happened, you'd then need to convince some distro
that enough people would care about cache lines and wanted the
additional performance so that they'd go through the hassle of
building this and QAing, etc. You'd also have to hack on any
multimedia libraries and crypto libraries to provide the optimised
versions to get decent performance.

I have a vague feeling that we've heard from the only person who
*actually* cares about this problem. Anyone else is probably using
ppc+altivec, sparc or some other arch where the 32 bit environment in
closer to what the 64 bit one provides.

Tks,
Jeff Bailey

--
Jeff Bailey - http://www.raspberryginger.com/jbailey/
 - "Remember, homosexuality is a choice, like cancer" - midwestteensexshow.com

[Josh Haberman (jhaberman)]

I know it's been a couple years since the bug was closed, but I found it today when wishing for a similar thing (and wondering if anyone had taken up Knuth on his idea).

FWIW I think you are significantly overstating what it would take to do this. I think this is comparable in scope to the -malign-double flag, which likewise changes the ABI slightly, but doesn't require a new ABI document, a new GNU triple, or changes to binutils.

GCC already knows how to compile for x86-64, the only difference would be that it would only allocate 32 bits for pointers in structs, and it would load/store those pointers using DWORD PTR instead of QWORD PTR. Besides that, the ABI would remain unchanged (for example, there would be no impact on calling conventions).

It's true that you'd need to get the kernel to allocate the stack and vdso within 4GB, but since the kernel already does this for compatibility mode binaries (ie. 32-bit binaries run on a 64-bit OS), I can't imagine it would be too difficult to accomplish.

The only big problem I can see is compatibility with kernel/glibc structures that have pointers in them, like "struct iovec"

           struct iovec {
               void *iov_base; /* Starting address */
               size_t iov_len; /* Number of bytes to transfer */
           };

To use structures like this you'd need to compile a separate glibc that uses 32-bit pointers. I'm not sure how -malign-double gets around this problem; maybe there just aren't any glibc/OS structs that have doubles in them. But I don't think too many of the structs that are exchanged between user code and glibc/kernel have pointers in them, so perhaps this problem wouldn't be too wide-spread.

Anyway, I'm not commenting on whether Ubuntu is the right place to make this feature request, I'm just commenting on the feasibility of doing this. I think it's a lot easier than you're making it sound.

[Josh Haberman (jhaberman)]

Also I don't think Knuth is the only one who could possibly care about this: some binaries actually run slower on x86-64 than on x86 (up to 10% from numbers I've seen) and the only possible reason for this slowdown AFAIK is the bigger pointers on x86-64 and the resulting effects on dcache and icache.