StaticScheduler has not attribute schedule

[Michal Arbet]

Hi,

When we are trying to use new StaticScheduler proposed in [0], it's failing, please check log error below :

BGP dynamic routing agent 2022-06-29 10:00:49 controller1
2022-06-29 10:07:06.160 25 INFO neutron.wsgi [-] 192.168.205.10 "GET / HTTP/1.1" status: 200 len: 232 time: 0.0052805
2022-06-29 10:07:10.276 34 WARNING neutron.db.agentschedulers_db [req-8d9ed03d-67bd-4685-9697-69ca1e04214a - - - - -] Rescheduling bgp_speaker de024f06-f373-4e5f-8c60-02e2e4e737a9 from agent dd10802c-b5a2-4673-84d2-a26aa09199fc because the agent did not report to the server in the last 150 seconds.
2022-06-29 10:07:10.311 34 ERROR neutron.db.agentschedulers_db [req-8d9ed03d-67bd-4685-9697-69ca1e04214a - - - - -] Exception encountered during bgp_speaker rescheduling.: AttributeError: 'StaticScheduler' object has no attribute 'schedule'
2022-06-29 10:07:10.311 34 ERROR neutron.db.agentschedulers_db Traceback (most recent call last):
2022-06-29 10:07:10.311 34 ERROR neutron.db.agentschedulers_db File "/var/lib/kolla/venv/lib/python3.9/site-packages/neutron/db/agentschedulers_db.py", line 172, in reschedule_resources_from_down_agents
2022-06-29 10:07:10.311 34 ERROR neutron.db.agentschedulers_db reschedule_resource(context, binding_resource_id)
2022-06-29 10:07:10.311 34 ERROR neutron.db.agentschedulers_db File "/var/lib/kolla/venv/lib/python3.9/site-packages/neutron_dynamic_routing/db/bgp_dragentscheduler_db.py", line 168, in reschedule_bgp_speaker
2022-06-29 10:07:10.311 34 ERROR neutron.db.agentschedulers_db self.schedule_bgp_speaker(context, bgp_speaker)
2022-06-29 10:07:10.311 34 ERROR neutron.db.agentschedulers_db File "/var/lib/kolla/venv/lib/python3.9/site-packages/neutron_dynamic_routing/db/bgp_dragentscheduler_db.py", line 99, in schedule_bgp_speaker
2022-06-29 10:07:10.311 34 ERROR neutron.db.agentschedulers_db agents = self.bgp_drscheduler.schedule(self, context,
2022-06-29 10:07:10.311 34 ERROR neutron.db.agentschedulers_db AttributeError: 'StaticScheduler' object has no attribute 'schedule'
2022-06-29 10:07:10.311 34 ERROR neutron.db.agentschedulers_db
2022-06-29 10:07:36.434 25 INFO neutron.wsgi [-] 192.168.205.10 "GET / HTTP/1.1" status: 200 len: 232 time: 0.0038111

[0] https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/815265

Could you please fix it ?

Thanks,
Kevko

[Michal Arbet (michalarbet)]

Additional info how to reproduce bug :

2 Speakers
2 Dragents

Reproduce :

1.

Configure bgp_drscheduler_driver in neutron.conf (neutron-server) as below :

[DEFAULT]
bgp_drscheduler_driver = neutron_dynamic_routing.services.bgp.scheduler.bgp_dragent_scheduler.StaticScheduler

2.

Add Speaker 1 -> dragent 1
Add Speaker 2 -> dragent 2

3.

Turn off speaker 2 for example.

3. Check neutron server log.

[Dr. Jens Harbott (j-harbott)]

How do you "Turn off speaker 2"? Do you talk about stopping the dragent?

Also how are your speakers configured that you need two of them? A standard deployment would have only one speaker scheduled to both agents.

[Michal Arbet (michalarbet)]

(openstack) bgp speaker list
+--------------------------------------+-------------+----------+------------+
| ID | Name | Local AS | IP Version |
+--------------------------------------+-------------+----------+------------+
| 165c1eac-2bc6-422b-9ff1-ae6c396f482c | bgpspeaker | 123 | 4 |
+--------------------------------------+-------------+----------+------------+
(openstack) bgp dragent list
+--------------------------------------+-------------+-------+-------+
| ID | Host | State | Alive |
+--------------------------------------+-------------+-------+-------+
| 937e9726-1e6a-48cb-abe0-652809cd6bf8 | controller0 | True | :-) |
| dd10802c-b5a2-4673-84d2-a26aa09199fc | controller1 | True | :-) |
+--------------------------------------+-------------+-------+-------+

(openstack) bgp speaker show dragents 165c1eac-2bc6-422b-9ff1-ae6c396f482c
The 'openstack bgp speaker show dragents' CLI is deprecated and will be removed in the future. Use 'openstack bgp dragent list' CLI instead.

(openstack) bgp speaker show dragents de024f06-f373-4e5f-8c60-02e2e4e737a9
The 'openstack bgp speaker show dragents' CLI is deprecated and will be removed in the future. Use 'openstack bgp dragent list' CLI instead.

(openstack) bgp dragent add speaker 937e9726-1e6a-48cb-abe0-652809cd6bf8 165c1eac-2bc6-422b-9ff1-ae6c396f482c
(openstack) bgp dragent add speaker dd10802c-b5a2-4673-84d2-a26aa09199fc 165c1eac-2bc6-422b-9ff1-ae6c396f482c
(openstack) bgp speaker show dragents 165c1eac-2bc6-422b-9ff1-ae6c396f482c
The 'openstack bgp speaker show dragents' CLI is deprecated and will be removed in the future. Use 'openstack bgp dragent list' CLI instead.
+--------------------------------------+-------------+-------+-------+
| ID | Host | State | Alive |
+--------------------------------------+-------------+-------+-------+
| 937e9726-1e6a-48cb-abe0-652809cd6bf8 | controller0 | True | :-) |
| dd10802c-b5a2-4673-84d2-a26aa09199fc | controller1 | True | :-) |
+--------------------------------------+-------------+-------+-------+

So, now I have one speaker scheduled to both controllers as you said this is standard deployment.
Then, if I go to one of controllers and just call docker stop neutron_bgp_dragent. I can see error below :

So, it doesn't matter if you have N speakers or just one. Behaviour is same:

2022-06-29 13:33:48.366 35 WARNING neutron.db.agentschedulers_db [req-be677f3a-c39a-4f43-82a2-cf2184e0412d - - - - -] Rescheduling bgp_speaker 165c1eac-2bc6-422b-9ff1-ae6c396f482c from agent 937e9726-1e6a-48cb-abe0-652809cd6bf8 because the agent did not report to the server in the last 150 seconds.
2022-06-29 13:33:48.419 35 ERROR neutron.db.agentschedulers_db [req-be677f3a-c39a-4f43-82a2-cf2184e0412d - - - - -] Exception encountered during bgp_speaker rescheduling.: AttributeError: 'StaticScheduler' object has no attribute 'schedule'
2022-06-29 13:33:48.419 35 ERROR neutron.db.agentschedulers_db Traceback (most recent call last):
2022-06-29 13:33:48.419 35 ERROR neutron.db.agentschedulers_db File "/var/lib/kolla/venv...

[Michal Arbet (michalarbet)]

And yeah, sorry, when I was talking about turn off speaker , I of course meant dragent.

[Michal Arbet (michalarbet)]

Any news ?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/849178

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/849178
Committed: https://opendev.org/openstack/neutron-dynamic-routing/commit/012f2358d23ef72518ef9bdef8a39f31b1fb86e2
Submitter: "Zuul (22348)"
Branch: master

commit 012f2358d23ef72518ef9bdef8a39f31b1fb86e2
Author: Dr. Jens Harbott <email address hidden>
Date: Sat Jul 9 09:45:45 2022 +0200

    Fix missing functions in StaticScheduler

    Seems the original approach was too simple, it is causing some
    tracebacks due to missing functions.

    Closes-Bug: 1980235
    Change-Id: I15ef7ec34eeccf4dfcce94b3bca2c853c4c4a609

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/xena
Review: https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/852109

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/yoga
Review: https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/852499

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/852501

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/852109
Committed: https://opendev.org/openstack/neutron-dynamic-routing/commit/75c5b9537e8fa070f7d45e335e0482db7e32a8f1
Submitter: "Zuul (22348)"
Branch: stable/xena

commit 75c5b9537e8fa070f7d45e335e0482db7e32a8f1
Author: Dr. Jens Harbott <email address hidden>
Date: Sat Jul 9 09:45:45 2022 +0200

    Fix missing functions in StaticScheduler

    Seems the original approach was too simple, it is causing some
    tracebacks due to missing functions.

    Closes-Bug: 1980235
    Change-Id: I15ef7ec34eeccf4dfcce94b3bca2c853c4c4a609
    (cherry picked from commit 012f2358d23ef72518ef9bdef8a39f31b1fb86e2)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/852499
Committed: https://opendev.org/openstack/neutron-dynamic-routing/commit/97853cbcb566b709c23abff822bc613f17d74ec1
Submitter: "Zuul (22348)"
Branch: stable/yoga

commit 97853cbcb566b709c23abff822bc613f17d74ec1
Author: Dr. Jens Harbott <email address hidden>
Date: Sat Jul 9 09:45:45 2022 +0200

    Fix missing functions in StaticScheduler

    Seems the original approach was too simple, it is causing some
    tracebacks due to missing functions.

    Closes-Bug: 1980235
    Change-Id: I15ef7ec34eeccf4dfcce94b3bca2c853c4c4a609
    (cherry picked from commit 012f2358d23ef72518ef9bdef8a39f31b1fb86e2)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/852501
Committed: https://opendev.org/openstack/neutron-dynamic-routing/commit/5371d7157a53f73f7dc043ba5163852e517674d7
Submitter: "Zuul (22348)"
Branch: stable/wallaby

commit 5371d7157a53f73f7dc043ba5163852e517674d7
Author: Dr. Jens Harbott <email address hidden>
Date: Sat Jul 9 09:45:45 2022 +0200

    Fix missing functions in StaticScheduler

    Seems the original approach was too simple, it is causing some
    tracebacks due to missing functions.

    Closes-Bug: 1980235
    Change-Id: I15ef7ec34eeccf4dfcce94b3bca2c853c4c4a609
    (cherry picked from commit 012f2358d23ef72518ef9bdef8a39f31b1fb86e2)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron-dynamic-routing 21.0.0.0rc1 release candidate.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron-dynamic-routing 18.1.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron-dynamic-routing 19.1.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron-dynamic-routing 20.0.1 release.