I've looked as some /proc/meminfo dumps I was provided with.
Dump were made at these time points:
1. after boot
2. successful core refresh from 2.38 to 2.37.4
3. successful core refresh from 2.37.4 to 2.38
4. failed refresh of other snap, failed when mounting the snap with: fork/exec /usr/bin/unsquashfs: cannot allocate memory
This is a list of values that varied between dumps (with the exception of LowTotal and CmaTotal): (1) (2) (3) (4)
Interesting observations:
- MemAvailable stays high, consistent with top output recordings I was provided with previously
- CmaFree drops, that's the memory available to the contiguous memory allocation, IIRC that's for DMA transfers and such
- Slab unreclaimable (SUnreclaim) rises
Even though MemAvailable is high, launching new processes fails. CMA free falling is intersting, I would expect weird things to happen when it drops too low.
I think it would be worth collecting the output of:
- sudo cat /proc/slabinfo
- sudo slabtop -o
In the kernel source code, in tools/vm, there is a tool `slabinfo` which can process the data from /proc/slabinfo in a useful manner. If possible, I think it would be useful to rebuild the tool for given architecture and collect the output of:
- slabinfo
- slabinfo -T
Another angle that may be interesting to explore, just for the sake of making sure we check for the obvious things, is to collect this:
- cat /proc/sys/kernel/pid_max
- ps -efT | wc -l
- dmesg
I've looked as some /proc/meminfo dumps I was provided with.
Dump were made at these time points: unsquashfs: cannot allocate memory
1. after boot
2. successful core refresh from 2.38 to 2.37.4
3. successful core refresh from 2.37.4 to 2.38
4. failed refresh of other snap, failed when mounting the snap with: fork/exec /usr/bin/
This is a list of values that varied between dumps (with the exception of LowTotal and CmaTotal):
(1) (2) (3) (4)
MemFree: 4536 kB 11068 kB 23088 kB 52928 kB
MemAvailable: 163016 kB 163572 kB 163900 kB 155532 kB
Buffers: 29036 kB 30404 kB 15008 kB 1992 kB
Cached: 131148 kB 123692 kB 128144 kB 104288 kB
Active: 110720 kB 109724 kB 102740 kB 129208 kB
Inactive: 80724 kB 75892 kB 71808 kB 16128 kB
Active(anon): 33848 kB 34136 kB 33984 kB 41660 kB
Inactive(anon): 3160 kB 3160 kB 3160 kB 3160 kB
Active(file): 76872 kB 75588 kB 68756 kB 87548 kB
Inactive(file): 77564 kB 72732 kB 68648 kB 12968 kB
LowTotal: 243168 kB 243168 kB 243168 kB 243168 kB
LowFree: 4536 kB 11068 kB 23088 kB 52928 kB
AnonPages: 31304 kB 31568 kB 31416 kB 39092 kB
Mapped: 54896 kB 55140 kB 52052 kB 18556 kB
Slab: 23968 kB 23340 kB 22424 kB 21780 kB
SReclaimable: 11112 kB 11252 kB 10476 kB 9156 kB
SUnreclaim: 12856 kB 12088 kB 11948 kB 12624 kB
KernelStack: 1032 kB 1000 kB 992 kB 1008 kB
PageTables: 1064 kB 1044 kB 1044 kB 1060 kB
Committed_AS: 489144 kB 339420 kB 487848 kB 512848 kB
CmaTotal: 65536 kB 65536 kB 65536 kB 65536 kB
CmaFree: 2128 kB 208 kB 15752 kB 33700 kB
Interesting observations:
- MemAvailable stays high, consistent with top output recordings I was provided with previously
- CmaFree drops, that's the memory available to the contiguous memory allocation, IIRC that's for DMA transfers and such
- Slab unreclaimable (SUnreclaim) rises
Even though MemAvailable is high, launching new processes fails. CMA free falling is intersting, I would expect weird things to happen when it drops too low.
I think it would be worth collecting the output of:
- sudo cat /proc/slabinfo
- sudo slabtop -o
In the kernel source code, in tools/vm, there is a tool `slabinfo` which can process the data from /proc/slabinfo in a useful manner. If possible, I think it would be useful to rebuild the tool for given architecture and collect the output of:
- slabinfo
- slabinfo -T
Another angle that may be interesting to explore, just for the sake of making sure we check for the obvious things, is to collect this: kernel/ pid_max
- cat /proc/sys/
- ps -efT | wc -l
- dmesg