Sure, here are the DEBUG logs from Neutron API for the non-working case with a distributed router.
It's just the moment when running
openstack -vv firewall group set --port 04222c40-dde1-4887-9fc2-fcf42a035fd4 oh_noes
2018-05-03 15:25:01.414 28360 DEBUG neutron.wsgi [-] (28360) accepted ('10.0.6.171', 42152) server /usr/lib/python2.7/site-packages/eventlet/wsgi.py:867
2018-05-03 15:25:01.419 28360 WARNING keystonemiddleware.auth_token [-] Using the in-process token cache is deprecated as of the 4.2.0 release and may be removed in the 5.0.0 release or the 'O' development cycle. The in-process cache causes inconsistent results and high memory usage. When the feature is removed the auth_token middleware will not cache tokens by default which may result in performance issues. It is recommended to use memcache for the auth_token token cache by setting the memcached_servers option.
2018-05-03 15:25:01.841 28360 DEBUG oslo_policy._cache_handler [req-cbd43c29-9637-443e-92cd-865fd2efcf64 ceeebe28f43b415a8c4059c414a1574e 2ca9201df90c4367a178d72426231509 - - -] Reloading cached file /etc/neutron/policy.json read_cached_file /usr/lib/python2.7/site-packages/oslo_policy/_cache_handler.py:40
2018-05-03 15:25:01.865 28360 DEBUG oslo_policy.policy [req-cbd43c29-9637-443e-92cd-865fd2efcf64 ceeebe28f43b415a8c4059c414a1574e 2ca9201df90c4367a178d72426231509 - - -] Reloaded policy file: /etc/neutron/policy.json _load_policy_file /usr/lib/python2.7/site-packages/oslo_policy/policy.py:666
2018-05-03 15:25:02.078 28360 DEBUG neutron_fwaas.db.firewall.v2.firewall_db_v2 [req-cbd43c29-9637-443e-92cd-865fd2efcf64 ceeebe28f43b415a8c4059c414a1574e 2ca9201df90c4367a178d72426231509 - - -] get_firewall_groups() called get_firewall_groups /usr/lib/python2.7/site-packages/neutron_fwaas/db/firewall/v2/firewall_db_v2.py:812
2018-05-03 15:25:02.111 28360 INFO neutron.wsgi [req-cbd43c29-9637-443e-92cd-865fd2efcf64 ceeebe28f43b415a8c4059c414a1574e 2ca9201df90c4367a178d72426231509 - - -] 10.0.6.171 "GET /v2.0/fwaas/firewall_groups.json?name=oh_noes HTTP/1.1" status: 200 len: 552 time: 0.6945391
2018-05-03 15:25:02.221 28360 INFO neutron.wsgi [req-fa1c064f-3706-473d-8f4f-d57eb334bf7f ceeebe28f43b415a8c4059c414a1574e 2ca9201df90c4367a178d72426231509 - - -] 10.0.6.171 "GET /v2.0/ports.json?id=04222c40-dde1-4887-9fc2-fcf42a035fd4 HTTP/1.1" status: 200 len: 1238 time: 0.1052740
2018-05-03 15:25:02.239 28360 DEBUG neutron_fwaas.db.firewall.v2.firewall_db_v2 [req-41c572e3-6b53-4be9-9218-a67ff5d54d96 ceeebe28f43b415a8c4059c414a1574e 2ca9201df90c4367a178d72426231509 - - -] get_firewall_groups() called get_firewall_groups /usr/lib/python2.7/site-packages/neutron_fwaas/db/firewall/v2/firewall_db_v2.py:812
2018-05-03 15:25:02.247 28360 INFO neutron.wsgi [req-41c572e3-6b53-4be9-9218-a67ff5d54d96 ceeebe28f43b415a8c4059c414a1574e 2ca9201df90c4367a178d72426231509 - - -] 10.0.6.171 "GET /v2.0/fwaas/firewall_groups.json?name=oh_noes HTTP/1.1" status: 200 len: 552 time: 0.0208750
2018-05-03 15:25:02.276 28360 DEBUG oslo_messaging._drivers.amqpdriver [req-8a8a320b-659e-4364-9604-d41e0b04d6ea ceeebe28f43b415a8c4059c414a1574e 2ca9201df90c4367a178d72426231509 - - -] CAST unique_id: a99da93ddac747dc868b565bb418edf2 NOTIFY exchange 'neutron' topic 'notifications.info' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:534
2018-05-03 15:25:02.283 28360 DEBUG neutron.api.v2.base [req-8a8a320b-659e-4364-9604-d41e0b04d6ea ceeebe28f43b415a8c4059c414a1574e 2ca9201df90c4367a178d72426231509 - - -] Request body: {u'firewall_group': {u'ports': [u'04222c40-dde1-4887-9fc2-fcf42a035fd4']}} prepare_request_body /usr/lib/python2.7/site-packages/neutron/api/v2/base.py:698
2018-05-03 15:25:02.283 28360 DEBUG neutron_fwaas.db.firewall.v2.firewall_db_v2 [req-8a8a320b-659e-4364-9604-d41e0b04d6ea ceeebe28f43b415a8c4059c414a1574e 2ca9201df90c4367a178d72426231509 - - -] get_firewall_group() called get_firewall_group /usr/lib/python2.7/site-packages/neutron_fwaas/db/firewall/v2/firewall_db_v2.py:807
2018-05-03 15:25:02.291 28360 DEBUG neutron_fwaas.services.firewall.fwaas_plugin_v2 [req-8a8a320b-659e-4364-9604-d41e0b04d6ea ceeebe28f43b415a8c4059c414a1574e 2ca9201df90c4367a178d72426231509 - - -] update_firewall_group() called on firewall_group c2cff5a9-22dd-463d-a074-ea1551bcc116 update_firewall_group /usr/lib/python2.7/site-packages/neutron_fwaas/services/firewall/fwaas_plugin_v2.py:279
2018-05-03 15:25:02.291 28360 DEBUG neutron_fwaas.db.firewall.v2.firewall_db_v2 [req-8a8a320b-659e-4364-9604-d41e0b04d6ea ceeebe28f43b415a8c4059c414a1574e 2ca9201df90c4367a178d72426231509 - - -] get_firewall_group() called get_firewall_group /usr/lib/python2.7/site-packages/neutron_fwaas/db/firewall/v2/firewall_db_v2.py:807
2018-05-03 15:25:02.354 28360 INFO neutron.api.v2.resource [req-8a8a320b-659e-4364-9604-d41e0b04d6ea ceeebe28f43b415a8c4059c414a1574e 2ca9201df90c4367a178d72426231509 - - -] update failed (client error): There was a conflict when trying to complete your request.
2018-05-03 15:25:02.354 28360 INFO neutron.wsgi [req-8a8a320b-659e-4364-9604-d41e0b04d6ea ceeebe28f43b415a8c4059c414a1574e 2ca9201df90c4367a178d72426231509 - - -] 10.0.6.171 "PUT /v2.0/fwaas/firewall_groups/c2cff5a9-22dd-463d-a074-ea1551bcc116.json HTTP/1.1" status: 409 len: 355 time: 0.1014690
Sure, here are the DEBUG logs from Neutron API for the non-working case with a distributed router. dde1-4887- 9fc2-fcf42a035f d4 oh_noes
It's just the moment when running
openstack -vv firewall group set --port 04222c40-
2018-05-03 15:25:01.414 28360 DEBUG neutron.wsgi [-] (28360) accepted ('10.0.6.171', 42152) server /usr/lib/ python2. 7/site- packages/ eventlet/ wsgi.py: 867 are.auth_ token [-] Using the in-process token cache is deprecated as of the 4.2.0 release and may be removed in the 5.0.0 release or the 'O' development cycle. The in-process cache causes inconsistent results and high memory usage. When the feature is removed the auth_token middleware will not cache tokens by default which may result in performance issues. It is recommended to use memcache for the auth_token token cache by setting the memcached_servers option. _cache_ handler [req-cbd43c29- 9637-443e- 92cd-865fd2efcf 64 ceeebe28f43b415 a8c4059c414a157 4e 2ca9201df90c436 7a178d724262315 09 - - -] Reloading cached file /etc/neutron/ policy. json read_cached_file /usr/lib/ python2. 7/site- packages/ oslo_policy/ _cache_ handler. py:40 9637-443e- 92cd-865fd2efcf 64 ceeebe28f43b415 a8c4059c414a157 4e 2ca9201df90c436 7a178d724262315 09 - - -] Reloaded policy file: /etc/neutron/ policy. json _load_policy_file /usr/lib/ python2. 7/site- packages/ oslo_policy/ policy. py:666 fwaas.db. firewall. v2.firewall_ db_v2 [req-cbd43c29- 9637-443e- 92cd-865fd2efcf 64 ceeebe28f43b415 a8c4059c414a157 4e 2ca9201df90c436 7a178d724262315 09 - - -] get_firewall_ groups( ) called get_firewall_groups /usr/lib/ python2. 7/site- packages/ neutron_ fwaas/db/ firewall/ v2/firewall_ db_v2.py: 812 9637-443e- 92cd-865fd2efcf 64 ceeebe28f43b415 a8c4059c414a157 4e 2ca9201df90c436 7a178d724262315 09 - - -] 10.0.6.171 "GET /v2.0/fwaas/ firewall_ groups. json?name= oh_noes HTTP/1.1" status: 200 len: 552 time: 0.6945391 3706-473d- 8f4f-d57eb334bf 7f ceeebe28f43b415 a8c4059c414a157 4e 2ca9201df90c436 7a178d724262315 09 - - -] 10.0.6.171 "GET /v2.0/ports. json?id= 04222c40- dde1-4887- 9fc2-fcf42a035f d4 HTTP/1.1" status: 200 len: 1238 time: 0.1052740 fwaas.db. firewall. v2.firewall_ db_v2 [req-41c572e3- 6b53-4be9- 9218-a67ff5d54d 96 ceeebe28f43b415 a8c4059c414a157 4e 2ca9201df90c436 7a178d724262315 09 - - -] get_firewall_ groups( ) called get_firewall_groups /usr/lib/ python2. 7/site- packages/ neutron_ fwaas/db/ firewall/ v2/firewall_ db_v2.py: 812 6b53-4be9- 9218-a67ff5d54d 96 ceeebe28f43b415 a8c4059c414a157 4e 2ca9201df90c436 7a178d724262315 09 - - -] 10.0.6.171 "GET /v2.0/fwaas/ firewall_ groups. json?name= oh_noes HTTP/1.1" status: 200 len: 552 time: 0.0208750 _drivers. amqpdriver [req-8a8a320b- 659e-4364- 9604-d41e0b04d6 ea ceeebe28f43b415 a8c4059c414a157 4e 2ca9201df90c436 7a178d724262315 09 - - -] CAST unique_id: a99da93ddac747d c868b565bb418ed f2 NOTIFY exchange 'neutron' topic 'notifications. info' _send /usr/lib/ python2. 7/site- packages/ oslo_messaging/ _drivers/ amqpdriver. py:534 659e-4364- 9604-d41e0b04d6 ea ceeebe28f43b415 a8c4059c414a157 4e 2ca9201df90c436 7a178d724262315 09 - - -] Request body: {u'firewall_group': {u'ports': [u'04222c40- dde1-4887- 9fc2-fcf42a035f d4']}} prepare_ request_ body /usr/lib/ python2. 7/site- packages/ neutron/ api/v2/ base.py: 698 fwaas.db. firewall. v2.firewall_ db_v2 [req-8a8a320b- 659e-4364- 9604-d41e0b04d6 ea ceeebe28f43b415 a8c4059c414a157 4e 2ca9201df90c436 7a178d724262315 09 - - -] get_firewall_ group() called get_firewall_group /usr/lib/ python2. 7/site- packages/ neutron_ fwaas/db/ firewall/ v2/firewall_ db_v2.py: 807 fwaas.services. firewall. fwaas_plugin_ v2 [req-8a8a320b- 659e-4364- 9604-d41e0b04d6 ea ceeebe28f43b415 a8c4059c414a157 4e 2ca9201df90c436 7a178d724262315 09 - - -] update_ firewall_ group() called on firewall_group c2cff5a9- 22dd-463d- a074-ea1551bcc1 16 update_ firewall_ group /usr/lib/ python2. 7/site- packages/ neutron_ fwaas/services/ firewall/ fwaas_plugin_ v2.py:279 fwaas.db. firewall. v2.firewall_ db_v2 [req-8a8a320b- 659e-4364- 9604-d41e0b04d6 ea ceeebe28f43b415 a8c4059c414a157 4e 2ca9201df90c436 7a178d724262315 09 - - -] get_firewall_ group() called get_firewall_group /usr/lib/ python2. 7/site- packages/ neutron_ fwaas/db/ firewall/ v2/firewall_ db_v2.py: 807 api.v2. resource [req-8a8a320b- 659e-4364- 9604-d41e0b04d6 ea ceeebe28f43b415 a8c4059c414a157 4e 2ca9201df90c436 7a178d724262315 09 - - -] update failed (client error): There was a conflict when trying to complete your request. 659e-4364- 9604-d41e0b04d6 ea ceeebe28f43b415 a8c4059c414a157 4e 2ca9201df90c436 7a178d724262315 09 - - -] 10.0.6.171 "PUT /v2.0/fwaas/ firewall_ groups/ c2cff5a9- 22dd-463d- a074-ea1551bcc1 16.json HTTP/1.1" status: 409 len: 355 time: 0.1014690
2018-05-03 15:25:01.419 28360 WARNING keystonemiddlew
2018-05-03 15:25:01.841 28360 DEBUG oslo_policy.
2018-05-03 15:25:01.865 28360 DEBUG oslo_policy.policy [req-cbd43c29-
2018-05-03 15:25:02.078 28360 DEBUG neutron_
2018-05-03 15:25:02.111 28360 INFO neutron.wsgi [req-cbd43c29-
2018-05-03 15:25:02.221 28360 INFO neutron.wsgi [req-fa1c064f-
2018-05-03 15:25:02.239 28360 DEBUG neutron_
2018-05-03 15:25:02.247 28360 INFO neutron.wsgi [req-41c572e3-
2018-05-03 15:25:02.276 28360 DEBUG oslo_messaging.
2018-05-03 15:25:02.283 28360 DEBUG neutron.api.v2.base [req-8a8a320b-
2018-05-03 15:25:02.283 28360 DEBUG neutron_
2018-05-03 15:25:02.291 28360 DEBUG neutron_
2018-05-03 15:25:02.291 28360 DEBUG neutron_
2018-05-03 15:25:02.354 28360 INFO neutron.
2018-05-03 15:25:02.354 28360 INFO neutron.wsgi [req-8a8a320b-