scripts/memory_test and threaded_memtest fail on arm

[Daniel Manrique]

On this arm system, which has access to 4 GB of RAM, running the default memory_test script results in this:

$ sudo /usr/share/checkbox/scripts/memory_test System Memory: 4037 MB
Free Memory: 3801 MB
Swap Memory: 1047 MB
Running Single Process Memory Test
Starting Threaded Memory Test
Running for 3991 MB total memory
Running for more than free memory at 3991 MB for 60 sec.
Command is: /usr/share/checkbox/scripts/threaded_memtest -qpv -m3991m -t60
/usr/share/checkbox/scripts/threaded_memtest returned code 1
More Than Free Memory Test failed

This helpfully reports the actual command used to try the run, so I tried that manually:

/usr/share/checkbox/scripts/threaded_memtest -qpv -m3991m -t60

$ /usr/share/checkbox/scripts/threaded_memtest -qpv -m3991m -t60
Warning: memsize > free_mem. You will probably hit swap.
Detected 4 processors.
RAM: 87.5% free (3531M/4037M)
Testing 3991M RAM for 60 seconds using 8 threads:
thread 1: mapping 498M RAM
thread 0: mapping 498M RAM
thread 2: mapping 498M RAM
thread 3: mapping 498M RAM
thread 4: mapping 498M RAM
thread 5: mapping 498M RAM
thread 6: mapping 498M RAM
thread 7: mapping 498M RAM
mmap: Cannot allocate memory
mmap: Cannot allocate memory

However reducing the memory to allocate works:

$ /usr/share/checkbox/scripts/threaded_memtest -qpv -m1991m -t5
Detected 4 processors.
RAM: 87.5% free (3532M/4037M)
Testing 1991M RAM for 5 seconds using 8 threads:
thread 1: mapping 248M RAM
<snip>
thread 7: mapping 248M RAM
thread 1: mapping complete
<snip>
thread 7: mapping complete
thread 4: test start
<snip>
thread 7: test start
thread 0 finished.
<snip>
thread 4 finished.
thread 4 unmapping and exiting
<snip>
thread 2 unmapping and exiting
Runtime was 5.63s
thread 0: 266353 loops
<snip>
thread 7: 293540 loops
Total loops per second: 406997.69
Testing complete.

By experimenting I found that the largest memory size is between 2844 and 2846 (it's variable, sometimes it works and sometimes it fails). I suspect it has to do with the executable being 32-bit and so unable to allocate more than this. Or maybe we need to revise the available memory calculations for arm.

[Brendan Donegan (brendan-donegan)]

Confirmed that process address space is limited to 32-bit on ARM. Interestingly this code exists in the memory_test script:

        # Process Memory
        self.process_memory = self.free_memory
        try:
            arch = self._command_out("arch").decode()
            if re.match(r"(i[0-9]86|s390)", arch) and self.free_memory > 1024:
                self.is_process_limited = True
                self.process_memory = 1024 # MB, due to 32-bit address space
                print("%s arch, Limiting Process Memory: %u" % (arch,
                    self.process_memory))

So perhaps this just needs to be made aware of ARM as an architecture. The 'arch' output on Calxeda is 'armv7l', so we could just add that in, but we might want to make the solution more flexible.

[Daniel Manrique (roadmr)]

How did you confirm the 32-bit limitation? is this something we can check programmatically, other than matching an architecture and assuming it's 32-bit? I'm thinking we don't want to limit ourselves when 64-bit ARM starts appearing.

[Zygmunt Krynicki (zyga)]

A15 has the equivalent of x86 PAE with 40 bit (IIRC) address lines. What are we exactly trying to guess here again?

[Brendan Donegan (brendan-donegan)]

I have a fix for this

[Christian Reis (kiko)]

Brendan, Daniel: oddly that patch's changelog notes that arm64 is excepted, but the code actually matches on 'arm.*' which doesn't seem to be what we want, right?

[Daniel Manrique (roadmr)]

Hi Christian,

The merge request's comment states "This works because the arch for ARM 64-bit is 'aarch64', so it won't match with 'arm.*'". We are under this assumption, do you know of any case where an arm64 device will have "arm.*" as its architecture, rather than "aarch64"?