With latest ISO, all baremetal configurations are passing sanity test (Green Status):
Regarding memory usage:
Before system is on "available" status:
+----+--------------+-------------+----------------+-------------+--------------+
| id | hostname | personality | administrative | operational | availability |
+----+--------------+-------------+----------------+-------------+--------------+
| 1 | controller-0 | controller | unlocked | disabled | intest |
+----+--------------+-------------+----------------+-------------+--------------+
total used free shared buff/cache available
Mem: 93G 5.5G 85G 45M 1.8G 86G
Swap: 0B 0B 0B
Less than 5 seconds after the system is on "available" status:
+----+--------------+-------------+----------------+-------------+--------------+
| id | hostname | personality | administrative | operational | availability |
+----+--------------+-------------+----------------+-------------+--------------+
| 1 | controller-0 | controller | unlocked | enabled | available |
+----+--------------+-------------+----------------+-------------+--------------+
total used free shared buff/cache available
Mem: 93G 72G 19G 46M 1.8G 20G
Swap: 0B 0B 0B
So it jumps from using 5.5GB to 72GB, when we reported the bug, the usage was 71GB, almost the same as today.
I'm assuming that docker reserves the memory because the pods/containers are not limited, as we can see on docker stats, almost all containers have their limit set by the total amount of physical memory on the system, i.e.:
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
e24823b3c8aa k8s_POD_testpod_default_4831b628-6a2b-11e9-9db4-3cfdfebedf30_0 0.00% 4.32MiB / 93.02GiB 0.00% 0B / 0B 0B / 0B 1
1afe1bd620c0 k8s_nova-compute-ssh_nova-compute-controller-0-a762cb46-9486m_openstack_8afe533b-6a2a-11e9-9db4-3cfdfebedf30_0 0.00% 12.51MiB / 93.02GiB 0.01% 0B / 0B 0B / 20.5kB 1
18d5e5ddb8f0 k8s_nova-compute_nova-compute-controller-0-a762cb46-9486m_openstack_8afe533b-6a2a-11e9-9db4-3cfdfebedf30_0 0.06% 197.2MiB / 93.02GiB 0.21% 0B / 0B 1.04GB / 135kB 22
Is this behavior expected? is there a way to properly track down memory usage at docker level?
With latest ISO, all baremetal configurations are passing sanity test (Green Status):
Regarding memory usage:
Before system is on "available" status: ------- ------+ ------- ------+ ------- ------- --+---- ------- --+---- ------- ---+ ------- ------+ ------- ------+ ------- ------- --+---- ------- --+---- ------- ---+ ------- ------+ ------- ------+ ------- ------- --+---- ------- --+---- ------- ---+
+----+-
| id | hostname | personality | administrative | operational | availability |
+----+-
| 1 | controller-0 | controller | unlocked | disabled | intest |
+----+-
total used free shared buff/cache available
Mem: 93G 5.5G 85G 45M 1.8G 86G
Swap: 0B 0B 0B
Less than 5 seconds after the system is on "available" status: ------- ------+ ------- ------+ ------- ------- --+---- ------- --+---- ------- ---+ ------- ------+ ------- ------+ ------- ------- --+---- ------- --+---- ------- ---+ ------- ------+ ------- ------+ ------- ------- --+---- ------- --+---- ------- ---+
+----+-
| id | hostname | personality | administrative | operational | availability |
+----+-
| 1 | controller-0 | controller | unlocked | enabled | available |
+----+-
total used free shared buff/cache available
Mem: 93G 72G 19G 46M 1.8G 20G
Swap: 0B 0B 0B
So it jumps from using 5.5GB to 72GB, when we reported the bug, the usage was 71GB, almost the same as today.
I'm assuming that docker reserves the memory because the pods/containers are not limited, as we can see on docker stats, almost all containers have their limit set by the total amount of physical memory on the system, i.e.:
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS testpod_ default_ 4831b628- 6a2b-11e9- 9db4-3cfdfebedf 30_0 0.00% 4.32MiB / 93.02GiB 0.00% 0B / 0B 0B / 0B 1 compute- ssh_nova- compute- controller- 0-a762cb46- 9486m_openstack _8afe533b- 6a2a-11e9- 9db4-3cfdfebedf 30_0 0.00% 12.51MiB / 93.02GiB 0.01% 0B / 0B 0B / 20.5kB 1 compute_ nova-compute- controller- 0-a762cb46- 9486m_openstack _8afe533b- 6a2a-11e9- 9db4-3cfdfebedf 30_0 0.06% 197.2MiB / 93.02GiB 0.21% 0B / 0B 1.04GB / 135kB 22
e24823b3c8aa k8s_POD_
1afe1bd620c0 k8s_nova-
18d5e5ddb8f0 k8s_nova-
Is this behavior expected? is there a way to properly track down memory usage at docker level?
Thanks!