| 2018-07-02 11:17:14 |
Pulkit Tandon |
bug |
|
|
added bug |
| 2018-07-02 11:17:14 |
Pulkit Tandon |
attachment added |
|
nodeh4_api_server.log https://bugs.launchpad.net/bugs/1779656/+attachment/5158623/+files/nodeh4_api_server.log |
|
| 2018-07-02 11:17:33 |
Pulkit Tandon |
nominated for series |
|
juniperopenstack/r5.0 |
|
| 2018-07-02 11:17:33 |
Pulkit Tandon |
bug task added |
|
juniperopenstack/r5.0 |
|
| 2018-07-02 11:17:33 |
Pulkit Tandon |
nominated for series |
|
juniperopenstack/trunk |
|
| 2018-07-02 11:17:33 |
Pulkit Tandon |
bug task added |
|
juniperopenstack/trunk |
|
| 2018-07-02 11:17:40 |
Pulkit Tandon |
juniperopenstack/r5.0: importance |
Undecided |
Critical |
|
| 2018-07-02 11:17:48 |
Pulkit Tandon |
juniperopenstack/r5.0: assignee |
|
Sachchidanand Vaidya (vaidyasd) |
|
| 2018-07-02 11:17:57 |
Pulkit Tandon |
juniperopenstack/r5.0: milestone |
|
r5.0.1 |
|
| 2018-07-02 11:19:51 |
Pulkit Tandon |
attachment added |
|
nodeh4_kube_manager.log https://bugs.launchpad.net/juniperopenstack/+bug/1779656/+attachment/5158624/+files/nodeh4_kube_manager.log |
|
| 2018-07-02 11:20:35 |
Pulkit Tandon |
summary |
[R5.0-k8s]: Network Policy which are part of "k8s-allowall" are getting deleted automatically. |
[R5.0-k8s]: Firewall rules which are part of "k8s-allowall" network policy are getting deleted automatically. |
|
| 2018-07-02 11:22:34 |
Pulkit Tandon |
description |
R5.0-117.
5 node plain k8s+contrail setup.
3 Controller
1 Kube master
2 Compute + k8s slave
Description:
As I create new project, I see that corresponding Allow all rules get added in "k8s-allowall" network policy.
But in some time, all policies corresponding to the namespaces which I created, get deleted automatically.
Thus, this result in test case failure.
I think this might be due to the name of the namespace used.
Prior to this test, I ran some other test cases which used same name for namespaces.
In the kube manager logs, I can see following Delete request running continuously.
07/02/2018 10:36:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace new-default:e5e05334-7dd5-11e8-a04b-002590aaa909
07/02/2018 10:36:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace temp-ns:e977c2a4-7dd5-11e8-a04b-002590aaa909
07/02/2018 10:37:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace non-default:e7abbdb5-7dd5-11e8-a04b-002590aaa909
07/02/2018 10:37:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace new-default:e5e05334-7dd5-11e8-a04b-002590aaa909
07/02/2018 10:37:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace temp-ns:e977c2a4-7dd5-11e8-a04b-002590aaa909
07/02/2018 10:38:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace non-default:e7abbdb5-7dd5-11e8-a04b-002590aaa909
07/02/2018 10:38:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace new-default:e5e05334-7dd5-11e8-a04b-002590aaa909
07/02/2018 10:38:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace temp-ns:e977c2a4-7dd5-11e8-a04b-002590aaa909
Note that the namespaces which I created are with same name "temp-ns", "new-default" and "non-default".
For current case, following is the namespace add request from the logs:
07/02/2018 10:11:10 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got ADDED Namespace temp-ns:3e4b542e-7de0-11e8-a04b-002590aaa909
I suspect that the stale Delete requests for same namespace name(but different UUID), resulted in deletion of network policies.
If that is the case, there are 2 things to inspect:
1. Why the stale namespace deletion request is running continuously?
2. Should this stale namespace deletion request result in deletion of network policies even though they have different UUID ?
For further verification, I did a restart of Kube manager.
All policies were restored after a restart as they were read fresh.
Soon after the stale DELETE, the policies were again deleted.
I have attached the kube manager and config logs. |
R5.0-117.
5 node plain k8s+contrail setup.
3 Controller
1 Kube master
2 Compute + k8s slave
Description:
As I create new project, I see that corresponding Allow all rules get added in "k8s-allowall" network policy.
But in some time, all rules corresponding to the namespaces which I created, get deleted automatically.
Thus, this result in test case failure.
I think this might be due to the name of the namespace used.
Prior to this test, I ran some other test cases which used same name for namespaces.
In the kube manager logs, I can see following Delete request running continuously.
07/02/2018 10:36:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace new-default:e5e05334-7dd5-11e8-a04b-002590aaa909
07/02/2018 10:36:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace temp-ns:e977c2a4-7dd5-11e8-a04b-002590aaa909
07/02/2018 10:37:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace non-default:e7abbdb5-7dd5-11e8-a04b-002590aaa909
07/02/2018 10:37:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace new-default:e5e05334-7dd5-11e8-a04b-002590aaa909
07/02/2018 10:37:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace temp-ns:e977c2a4-7dd5-11e8-a04b-002590aaa909
07/02/2018 10:38:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace non-default:e7abbdb5-7dd5-11e8-a04b-002590aaa909
07/02/2018 10:38:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace new-default:e5e05334-7dd5-11e8-a04b-002590aaa909
07/02/2018 10:38:43 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got DELETED Namespace temp-ns:e977c2a4-7dd5-11e8-a04b-002590aaa909
Note that the namespaces which I created are with same name "temp-ns", "new-default" and "non-default".
For current case, following is the namespace add request from the logs:
07/02/2018 10:11:10 AM [contrail-kube-manager] [DEBUG]: __default__ [SYS_DEBUG]: KubeManagerDebugLog: VncNamespace - Got ADDED Namespace temp-ns:3e4b542e-7de0-11e8-a04b-002590aaa909
I suspect that the stale Delete requests for same namespace name(but different UUID), resulted in deletion of network policies.
If that is the case, there are 2 things to inspect:
1. Why the stale namespace deletion request is running continuously?
2. Should this stale namespace deletion request result in deletion of network firewall rules even though they have different UUID ?
For further verification, I did a restart of Kube manager.
All rules were restored after a restart as they were read fresh.
Soon after the stale DELETE request, the rules were again deleted automatically.
I have attached the kube manager and config logs. |
|
| 2018-07-02 17:03:57 |
Sachchidanand Vaidya |
juniperopenstack/r5.0: assignee |
Sachchidanand Vaidya (vaidyasd) |
Dinesh Bakiaraj (dineshb) |
|
| 2018-07-02 17:04:06 |
Sachchidanand Vaidya |
juniperopenstack/trunk: assignee |
Sachchidanand Vaidya (vaidyasd) |
Dinesh Bakiaraj (dineshb) |
|
| 2018-07-02 23:16:06 |
Dinesh Bakiaraj |
juniperopenstack/r5.0: assignee |
Dinesh Bakiaraj (dineshb) |
Pragash Vijayaragavan (pvijayaragav) |
|
| 2018-07-02 23:16:20 |
Dinesh Bakiaraj |
juniperopenstack/trunk: assignee |
Dinesh Bakiaraj (dineshb) |
Pragash Vijayaragavan (pvijayaragav) |
|
| 2018-07-03 17:09:09 |
Dinesh Bakiaraj |
juniperopenstack/r5.0: assignee |
Pragash Vijayaragavan (pvijayaragav) |
Dinesh Bakiaraj (dineshb) |
|
| 2018-07-03 17:09:11 |
Dinesh Bakiaraj |
juniperopenstack/trunk: assignee |
Pragash Vijayaragavan (pvijayaragav) |
Dinesh Bakiaraj (dineshb) |
|
| 2018-07-03 17:10:33 |
Dinesh Bakiaraj |
juniperopenstack/r5.0: status |
New |
In Progress |
|
| 2018-07-03 17:10:36 |
Dinesh Bakiaraj |
juniperopenstack/trunk: status |
New |
In Progress |
|
| 2018-07-04 16:38:07 |
Dinesh Bakiaraj |
juniperopenstack/r5.0: status |
In Progress |
Fix Committed |
|
| 2018-07-04 16:38:12 |
Dinesh Bakiaraj |
juniperopenstack/trunk: status |
In Progress |
Fix Committed |
|
| 2018-07-10 04:13:19 |
Pulkit Tandon |
juniperopenstack/r5.0: status |
Fix Committed |
Fix Released |
|
| 2018-07-10 04:13:23 |
Pulkit Tandon |
juniperopenstack/trunk: status |
Fix Committed |
Fix Released |
|
