Contrail 2.21-14: contrail-svc-monitor stuck in initializing when scaling multi VNF heat stacks

[Deepak Jeyaraman]

Contrail 2.21-14

Seeing the contrail-svc-monitor is stuck between initializing and active

This is observed the 2nd time - first time, we had to reprovision the contrail config node. This has happened again, we noticed the only test done in the past day was mutli -vnf service chains scaling via heat engine.

This was okay for 30 NS (60 VNF) but when we brought up another 15 NS , nova didnt spawn.

Output:

root@ccra-06:~# heat stack-list | grep jsm | wc -l
45
root@ccra-06:~# nova list | grep default | wc -l
60

More output in the RSI.log

root@ccra-06:~# contrail-version
Package Version Build-ID | Repo | Package Name
-------------------------------------- ------------------------------ ----------------------------------
contrail-analytics 2.21.1-14 14
contrail-config 2.21.1-14 14
contrail-config-openstack 2.21.1-14 14
contrail-control 2.21.1-14 14
contrail-dns 2.21.1-14 14
contrail-f5 2.21.1-14 14

[Deepak Jeyaraman (jdeepak)]

Logs are stored here in svl shell:

/volume/vmcores/sagrawal/jdeepak_temp/NFV/contrail_bug/1527945

[Deepak Jeyaraman (jdeepak)]

Note:

This issue is seen only for multi-VNF NS chain (heat stack) and not a single VNF heat stack.