Comment 10 for bug 1826308

Revision history for this message
Cristopher Lemus (cjlemusc) wrote :

With latest ISO, all baremetal configurations are passing sanity test (Green Status):

Regarding memory usage:

Before system is on "available" status:
+----+--------------+-------------+----------------+-------------+--------------+
| id | hostname | personality | administrative | operational | availability |
+----+--------------+-------------+----------------+-------------+--------------+
| 1 | controller-0 | controller | unlocked | disabled | intest |
+----+--------------+-------------+----------------+-------------+--------------+
              total used free shared buff/cache available
Mem: 93G 5.5G 85G 45M 1.8G 86G
Swap: 0B 0B 0B

Less than 5 seconds after the system is on "available" status:
+----+--------------+-------------+----------------+-------------+--------------+
| id | hostname | personality | administrative | operational | availability |
+----+--------------+-------------+----------------+-------------+--------------+
| 1 | controller-0 | controller | unlocked | enabled | available |
+----+--------------+-------------+----------------+-------------+--------------+
              total used free shared buff/cache available
Mem: 93G 72G 19G 46M 1.8G 20G
Swap: 0B 0B 0B

So it jumps from using 5.5GB to 72GB, when we reported the bug, the usage was 71GB, almost the same as today.

I'm assuming that docker reserves the memory because the pods/containers are not limited, as we can see on docker stats, almost all containers have their limit set by the total amount of physical memory on the system, i.e.:

CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
e24823b3c8aa k8s_POD_testpod_default_4831b628-6a2b-11e9-9db4-3cfdfebedf30_0 0.00% 4.32MiB / 93.02GiB 0.00% 0B / 0B 0B / 0B 1
1afe1bd620c0 k8s_nova-compute-ssh_nova-compute-controller-0-a762cb46-9486m_openstack_8afe533b-6a2a-11e9-9db4-3cfdfebedf30_0 0.00% 12.51MiB / 93.02GiB 0.01% 0B / 0B 0B / 20.5kB 1
18d5e5ddb8f0 k8s_nova-compute_nova-compute-controller-0-a762cb46-9486m_openstack_8afe533b-6a2a-11e9-9db4-3cfdfebedf30_0 0.06% 197.2MiB / 93.02GiB 0.21% 0B / 0B 1.04GB / 135kB 22

Is this behavior expected? is there a way to properly track down memory usage at docker level?

Thanks!