Control Node - Streaming data from etcd or binglog to cassandra / rabbitmq

[Tomasz Młynarczyk]

Replicate etcd data to cassandra and rabbitmq for control node.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45064
Submitter: Tomasz Mlynarczyk (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45630
Submitter: Tomasz Mlynarczyk (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45630
Submitter: Bartlomiej Gajda (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45064
Submitter: Tomasz Mlynarczyk (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45064
Submitter: Jozef Woloch (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45064
Submitter: Tomasz Mlynarczyk (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45665
Submitter: Tomasz Mlynarczyk (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45064
Submitter: Jozef Woloch (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45630
Submitter: Tomasz Mlynarczyk (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45665
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45064
Submitter: Jozef Woloch (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45064
Submitter: Tomasz Mlynarczyk (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45630
Submitter: Tomasz Mlynarczyk (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45665
Committed: http://github.com/Juniper/contrail/commit/42381fcb71fe3e0e68af433daae05a7fa27efe2a
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 42381fcb71fe3e0e68af433daae05a7fa27efe2a
Author: Tomasz Młynarczyk <email address hidden>
Date: Mon Aug 20 17:29:41 2018 +0200

Support etcd update event

Identify update events using IsModified method from etcd event.

Change-Id: Ie8ae13d219a8cf3cb69f08f314fa703fc9fe3ff9
Partial-bug: #1784379

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45064
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45694
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45630
Submitter: Tomasz Mlynarczyk (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45708
Submitter: Daniel Furman (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45709
Submitter: Tomasz Mlynarczyk (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45630
Submitter: Tomasz Mlynarczyk (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45630
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45064
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45694
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45630
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45709
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45694
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45064
Committed: http://github.com/Juniper/contrail/commit/79e91c1dea92d0710ebfa10481b1f8dc69731aae
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 79e91c1dea92d0710ebfa10481b1f8dc69731aae
Author: Tomasz Młynarczyk <email address hidden>
Date: Mon Jul 30 10:59:58 2018 +0200

Etcd to cassandra replication service

replication service is needed for as long as control node can not read data from etcd directly.

Change-Id: Ib19c67814421e7793afcebb91dfdadb01da94059
Partial-bug: #1784379

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45630
Committed: http://github.com/Juniper/contrail/commit/205d45570543d9042766db487f1aadfcb076214c
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 205d45570543d9042766db487f1aadfcb076214c
Author: Tomasz Młynarczyk <email address hidden>
Date: Fri Aug 17 10:06:07 2018 +0200

etcd to rabbit replication

It is done in a separate service which is listening
on etcd events and produce msgs to rabbit.

Change-Id: Ic3f18bd2db28f0745c7ea1690df155a0fb10455f
Partial-bug: #1784379

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45709
Committed: http://github.com/Juniper/contrail/commit/812b69390c377a676da5a1bd06b06335e357d18f
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 812b69390c377a676da5a1bd06b06335e357d18f
Author: Tomasz Młynarczyk <email address hidden>
Date: Wed Aug 22 19:28:25 2018 +0200

Fix etcd update event data

Data was not being umarshalled for etcd update event.

Change-Id: Ie5bdb4bf937e7a4ca34e5f14935c0c8f6a7b6cce
Partial-bug: #1784379

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45694
Committed: http://github.com/Juniper/contrail/commit/c12fcf36905387c90f8609d8a4a00a69a35d4310
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit c12fcf36905387c90f8609d8a4a00a69a35d4310
Author: Michał Błotniak <email address hidden>
Date: Tue Aug 21 16:42:06 2018 -0700

Make Cassandra ConnectTimeout configurable

Default 600ms ConnectTimeout was too short for some connections (eg. VPN).

Change-Id: I7ccb85282a5670fec288bcd85bae11eeb3bad600
Partial-bug: #1784379

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45708
Submitter: Daniel Furman (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45730
Submitter: Tomasz Mlynarczyk (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45731
Submitter: Tomasz Mlynarczyk (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45730
Committed: http://github.com/Juniper/contrail/commit/75d69a9b441cb50b14c277e3222e047a99cd5ee0
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 75d69a9b441cb50b14c277e3222e047a99cd5ee0
Author: Tomasz Młynarczyk <email address hidden>
Date: Thu Aug 23 15:20:05 2018 +0200

Fix rabbit msg format

json.Marshal is marshaling []byte to base64 string
but we would like it to be a raw bytes.

Change-Id: I822413a5f1e4317151d52835f9b411da210e4158
Partial-bug: #1784379

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45731
Committed: http://github.com/Juniper/contrail/commit/f2785a805b3c8434a8c3b0c9f34cc8dcf0ef7165
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit f2785a805b3c8434a8c3b0c9f34cc8dcf0ef7165
Author: Tomasz Młynarczyk <email address hidden>
Date: Thu Aug 23 15:37:42 2018 +0200

Separate amqp and cass replicators

Allow separate enabling of amqp and cass replicators.

Change-Id: Id429f11d93d5cf452294e37a235d8e8fe937c62e
Partial-bug: #1784379

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45739
Submitter: Daniel Furman (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45743
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45773
Submitter: Daniel Furman (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45708
Submitter: Daniel Furman (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45739
Submitter: Daniel Furman (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45708
Submitter: Daniel Furman (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45739
Submitter: Daniel Furman (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45739
Committed: http://github.com/Juniper/contrail/commit/2023457e5eeb31828997a7c693d37e8547cb343c
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 2023457e5eeb31828997a7c693d37e8547cb343c
Author: Daniel Furman <email address hidden>
Date: Thu Aug 23 22:29:16 2018 +0200

Improve etcd notifier and and fix project's aps ref

- improve etcd notifier service
- add application-policy-set's fqName to ref on project create

Change-Id: I590566998cc5bca90b89dc96401d664a74ec98c4
Partial-Bug: 1784379

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45708
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45743
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45708
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45708
Committed: http://github.com/Juniper/contrail/commit/385c150d9ca55257c62c64b68fecc7884fee295e
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 385c150d9ca55257c62c64b68fecc7884fee295e
Author: Daniel Furman <email address hidden>
Date: Mon Aug 27 15:12:57 2018 +0200

Implement VNC API Notifier

Add VNC API Notifier to API Server.
It proxies all CUD requests to configured VNC API Server.

It is a workaround allowing using old VNC API Server as a Cassandra/Rabbit data writer.

Change-Id: I215971e5f1bcee2e0c0130cc6b9e680d4e467249
Partial-Bug: 1784379

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45743
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45824
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45743
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45824
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45743
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45824
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45743
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45824
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45743
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45824
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45743
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45824
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45743
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45743
Committed: http://github.com/Juniper/contrail/commit/6125bf0efa0a7d275fbcbff23b0efa8ea9ce3cb0
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit 6125bf0efa0a7d275fbcbff23b0efa8ea9ce3cb0
Author: Michał Błotniak <email address hidden>
Date: Thu Aug 23 17:08:21 2018 -0700

Create models.Codec and add some update utilities

Added helpers usefull in update cases:
- ApplyMap method that allows updating model objects using map
- GetFromMapByPath that gets values from nested maps
- ApplyFieldMask that filters map according to fieldmask

Change-Id: I9072a3b9d798a8052ecb65e88259fb9504a01b5d
Partial-Bug: 1784379

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/45824
Submitter: Michal Blotniak (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/45824
Committed: http://github.com/Juniper/contrail/commit/fcfc48aaaedcfa1dc2fe2f0cd5d49f87db5dce25
Submitter: Zuul v3 CI (<email address hidden>)
Branch: master

commit fcfc48aaaedcfa1dc2fe2f0cd5d49f87db5dce25
Author: Michał Błotniak <email address hidden>
Date: Tue Aug 28 12:18:37 2018 -0700

Implement update in etcd.NotifierService

Fix bug with update not respecting fieldmask and refactor
etcd.NotifierService to use models.Codec. Implement models.UpdateData
function for getting update payload for model objects using provided
codec.

Change-Id: I9b6f514090451597ec94b4d259a7fc9eb0a82583
Partial-Bug: 1784379