vrouter ksync memory assertion failed

[Curtis Collicutt]

I'm deploying a single node all in one contrail/kolla instance via contrail-ansible-deployer. I had this working on or about April 3rd, but after had various issues with replicating the deployment.

The vrouter agent init container failed, but I manually added the vrouter kernel module. Now I get this error:

[root@tungsten-1 centos]# docker logs vrouter_vrouter-agent_1 | tail
INFO: Preparing /etc/contrail/contrail-lbaas-auth.conf
Config file </etc/contrail/contrail-vrouter-agent.conf> parsing completed.
log4cplus:ERROR No appenders could be found for logger (SANDESH).
log4cplus:ERROR Please initialize the log4cplus system properly.
contrail-vrouter-agent: controller/src/vnsw/agent/vrouter/ksync/ksync_memory.cc:107: void KSyncMemory::Mmap(bool): Assertion `0' failed.
/entrypoint.sh: line 248: 20495 Aborted (core dumped) "$@"

Kernel info:

[root@tungsten-1 centos]# uname -a
Linux tungsten-1.local 3.10.0-693.21.1.el7.x86_64 #1 SMP Wed Mar 7 19:03:37 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@tungsten-1 centos]# lsmod | grep vrouter
vrouter 460243 0

OS:

[centos@tungsten-1 ~]$ cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)

NOTE:

Am using devicemapper as the docker storage backend due to issues with overlayfs and cpu lock ups and df hang.

[Hari Prasad Killi (haripk)]

Looks to me like some memory allocation issue in vrouter.

When vrouter-agent comes up, it fetches flow table size info from vrouter and it looks like vrouter didnt get memory allocated (it needs contiguous memory and that may have failed). A reboot of the compute node may avoid this situation.