Again : High CPU usage of Mysqld process with opendaylight journaling

[[e-mail]]

Hi,
  Recently, i met an issue that high CPU usage of Mysqld process with opendaylight journaling. So i search the solution by google and i found that there was already existed a bug report on networking-odl project(1717018 High CPU usage of Mysqld process with opendaylight journaling). But i found that the cause of this bug is differ from mine.
   I found that high CPU usage of Mysqld process with opendaylight journaling when i stop the opendaylight process. So the neutron resources are not able sync to opendaylight. As time goes on, the journal db rows growing up that result in the high CPU usage of Mysqld process.
   Below is my journal db rows count:
MySQL [neutron]> select count(*) from opendaylightjournal;
+----------+
| count(*) |
+----------+
| 23941 |
+----------+
1 row in set (0.02 sec)

MySQL [neutron]> select count(*) from opendaylightjournal where state='pending';
+----------+
| count(*) |
+----------+
| 23941 |
+----------+
1 row in set (0.01 sec)

MySQL [neutron]> select count(*) from opendaylight_journal_deps;
+----------+
| count(*) |
+----------+
| 8380997 |
+----------+
1 row in set (1.49 sec)

[Lajos Katona (lajos-katona)]

Thanks for reporting this issue.
Could you please give some details about your environment (versions, etc...)
If I understand well the simple reproduction step is to stop ODL process?

[[e-mail] (525215713-i)]

Hi,
  Thank you for your timely feedback. The version of networking-odl is stable/queens.
  The reproduction step is to stop ODL process.

[Lajos Katona (lajos-katona)]

Hi, Which version of ODL do you use, Oxygen or Nitrogen? (see: https://docs.opendaylight.org/en/stable-nitrogen/submodules/netvirt/docs/user-guide/support.html)

[[e-mail] (525215713-i)]

the version of ODL is neno

[Lajos Katona (lajos-katona)]

I checked your issue on master with Sodium opendaylight.
I had no big journaldb, as it is only a test environment.

First what is important: if opendaylight is dead and neutron API is used than the journal will contain more and more entries.

What I expect is that after restarting opendaylight:
* if I do something on the API the journal is synced towards opendaylight. For me this worked.
* after maintenance_interval (300sec by default under ml2_odl confing section) the contents of the journal should be synced towards opendaylight. For some reason this one is not working.

I check how to fix the periodic sync, but as a workaround could you please check if any request on neutron API (after starting again opendaylight creating a network or port) starts the sync?

[[e-mail] (525215713-i)]

Thank you for your answer. The issue that you met that the contents of the journal do not sync to opendaylight after maintenance_interval. I think the full sync had been executed befroe, so when you restart opendaylight, the contents of the journal do not sync to opendaylight after maintenance_interval. For the appearance of no big journaldb, you can create 100 vms in bulk and delete them. The process can repeat several times.

[Lajos Katona (lajos-katona)]

Sorry for late answers, but I in Shanghai on the Summit&ptg, I try to allocate some time to dig to the bottom of this.