migrate_to_ml2 script doesn't work for Juno release

[Xu Han Peng]

The error looks like:

 Traceback (most recent call last):
                File "migrate_to_ml2.py", line 485, in <module>
                  main()
                File "migrate_to_ml2.py", line 481, in main
                  args.vxlan_udp_port)
                File "migrate_to_ml2.py", line 135, in __call__
                  self.define_ml2_tables(metadata)
              AttributeError: 'MigrateOpenvswitchToMl2_Juno' object has no attribute 'define_ml2_tables'

This is caused by define_ml2_tables is a method of BaseMigrateToMl2_IcehouseMixin but not MigrateOpenvswitchToMl2_Juno. We should make MigrateOpenvswitchToMl2_Juno based on BaseMigrateToMl2_IcehouseMixin to solve this problem.

[Kyle Mestery (mestery)]

We need this for Juno. Mark is working on a fix for master now, and I'll cherry-pick that back once it lands there today.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/126977

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/126977
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=dd35d64812fd0ad02e7a00432073d141049e7107
Submitter: Jenkins
Branch: master

commit dd35d64812fd0ad02e7a00432073d141049e7107
Author: Mark McClain <email address hidden>
Date: Wed Oct 8 15:38:19 2014 -0400

    update ml2_migration to reflect optional methods

    This change conditionally executes the schema methods for versions that
    support them.

    Change-Id: I8a51ac04e62dfcfe1e0a2e69a17664d154f0d1d7
    Closes-Bug: #1378732

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: proposed/juno
Review: https://review.openstack.org/127148

[Xu Han Peng (xuhanp)]

Dropping self.define_ml2_tables(metadata) it self for Juno won't work because we will find another error:

Traceback (most recent call last):
  File "migrate_to_ml2.py", line 485, in <module>
    main()
  File "migrate_to_ml2.py", line 481, in main
    args.vxlan_udp_port)
  File "migrate_to_ml2.py", line 142, in __call__
    self.migrate_network_segments(engine, metadata)
  File "migrate_to_ml2.py", line 163, in migrate_network_segments
    ml2_network_segments = metadata.tables['ml2_network_segments']
KeyError: 'ml2_network_segments'

metadata object won't have ml2_network_segments table if not executing define_ml2_tables.

[Mark McClain (markmcclain)]

Moving this out of RC2. There is a workaround for this issue, so it is not a release blocker.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Thierry Carrez (<email address hidden>) on branch: proposed/juno
Review: https://review.openstack.org/127148

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: feature/lbaasv2
Review: https://review.openstack.org/130864

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/130864
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=c089154a94e5872efc95eab33d3d0c9de8619fe4
Submitter: Jenkins
Branch: feature/lbaasv2

commit 62588957fbeccfb4f80eaa72bef2b86b6f08dcf8
Author: Kevin Benton <email address hidden>
Date: Wed Oct 22 13:04:03 2014 -0700

    Big Switch: Switch to TLSv1 in server manager

    Switch to TLSv1 for the connections to the backend
    controllers. The default SSLv3 is no longer considered
    secure.

    TLSv1 was chosen over .1 or .2 because the .1 and .2 weren't
    added until python 2.7.9 so TLSv1 is the only compatible option
    for py26.

    Closes-Bug: #1384487
    Change-Id: I68bd72fc4d90a102003d9ce48c47a4a6a3dd6e03

commit 17204e8f02fdad046dabdb8b31397289d72c877b
Author: OpenStack Proposal Bot <email address hidden>
Date: Wed Oct 22 06:20:15 2014 +0000

    Imported Translations from Transifex

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: I58db0476c810aa901463b07c42182eef0adb5114

commit d712663b99520e6d26269b0ca193527603178742
Author: Carl Baldwin <email address hidden>
Date: Mon Oct 20 21:48:42 2014 +0000

    Move disabling of metadata and ipv6_ra to _destroy_router_namespace

    I noticed that disable_ipv6_ra is called from the wrong place and that
    in some cases it was called with a bogus router_id because the code
    made an incorrect assumption about the context. In other case, it was
    never called because _destroy_router_namespace was being called
    directly. This patch moves the disabling of metadata and ipv6_ra in
    to _destroy_router_namespace to ensure they get called correctly and
    avoid duplication.

    Change-Id: Ia76a5ff4200df072b60481f2ee49286b78ece6c4
    Closes-Bug: #1383495

commit f82a5117f6f484a649eadff4b0e6be9a5a4d18bb
Author: OpenStack Proposal Bot <email address hidden>
Date: Tue Oct 21 12:11:19 2014 +0000

    Updated from global requirements

    Change-Id: Idcbd730f5c781d21ea75e7bfb15959c8f517980f

commit be6bd82d43fbcb8d1512d8eb5b7a106332364c31
Author: Angus Lees <email address hidden>
Date: Mon Aug 25 12:14:29 2014 +1000

    Remove duplicate import of constants module

    .. and enable corresponding pylint check now the only offending instance
    is fixed.

    Change-Id: I35a12ace46c872446b8c87d0aacce45e94d71bae

commit 9902400039018d77aa3034147cfb24ca4b2353f6
Author: rajeev <email address hidden>
Date: Mon Oct 13 16:25:36 2014 -0400

    Fix race condition on processing DVR floating IPs

    Fip namespace and agent gateway port can be shared by multiple dvr routers.
    This change uses a set as the control variable for these shared resources
    and ensures that Test and Set operation on the control variable are
    performed atomically so that race conditions do not occur among
    multiple threads processing floating IPs.
    Limitation: The scope of this change is limited to addressing the race
    condition described in the bug report. It may not address other issues
    such as pre-existing issue wit...

[Eugene Nikanorov (enikanorov)]

What is the correct state of this bug? I see commit to master on 2014-10-09

[Attila Fazekas (afazekas)]

The bug is not solved, back-porting the patch merged to the master does not helps.

No handlers could be found for logger "oslo.db.sqlalchemy.session"
Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/lib/python2.7/site-packages/neutron/db/migration/migrate_to_ml2.py", line 486, in <module>
    main()
  File "/usr/lib/python2.7/site-packages/neutron/db/migration/migrate_to_ml2.py", line 482, in main
    args.vxlan_udp_port)
  File "/usr/lib/python2.7/site-packages/neutron/db/migration/migrate_to_ml2.py", line 143, in __call__
    self.migrate_network_segments(engine, metadata)
  File "/usr/lib/python2.7/site-packages/neutron/db/migration/migrate_to_ml2.py", line 164, in migrate_network_segments
    ml2_network_segments = metadata.tables['ml2_network_segments']
KeyError: 'ml2_network_segments'

[Shiv Haris (shh)]

Hi Mark, Any ideas/suggestions how to make this bug progress further.

[Shiv Haris (shh)]

Hi Mark, do we really need this fixed for K3 or can we postpone for L

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/juno
Review: https://review.openstack.org/166725

[lee jian (leejian0612)]

In my comprehension, this script has two mainly use, one is to migrate the db from openvswitch plugin to ml2 plugin on icehouse release, and another is to migrate the db from openvswitch plugin on ichouse release to ml2 plugin on juno release.

1.I use devstack to intall an all-in-one openstack(icehouse) as my test environment, and the default core_plugin is openvswitch. I follow the comment in the migrate_to_ml2.py to move it to ml2, and it works for me, cmd is:
python -m neutron.db.migration.migrate_to_ml2 openvswitch mysql://root:lee0612@127.0.0.1/ovs_neutron?charset=utf8 --release icehouse --save-tables True

2.To uppgrade db from openvswitch on icehouse release to ml2 on juno release, we hava two ways:
    1)separate it in two steps, first migrate openvswitch plugin to ml2 plugin on icehouse, and then use neutron-db-manage to upgrade it to the latest head juno, this works, cmd is:

    python -m neutron.db.migration.migrate_to_ml2 openvswitch mysql://root:lee0612@127.0.0.1/ovs_neutron?charset=utf8 --release icehouse --save-tables True
    neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head
    2) upgrade the db from icehouse to juno first, and then migrate openvswitch to ml2, this is how we get this bug. and I have a little confusion on this solution, as we know, juno do not hava openvswith plugin and linuxbridge plugin, they hava been replaced by ml2, so migrating from openvswitch, which is not exist, to ml2 on juno seems odd. Cmd is:

    neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head
    python -m neutron.db.migration.migrate_to_ml2 openvswitch mysql://root:lee0612@127.0.0.1/ovs_neutron?charset=utf8 --release juno --save-tables True(this failed)

    I also push a patch to stable/juno to solve this bug.
    https://review.openstack.org/#/c/166725/

[OpenStack Infra (hudson-openstack)]

Change abandoned by Jian LI (<email address hidden>) on branch: stable/juno
Review: https://review.openstack.org/166725
Reason: This patch should be commit to the master branch.

[Kyle Mestery (mestery)]

Is this still a relevant issue for folks?

[Kyle Mestery (mestery)]

Moving priority down a notch.

[Armando Migliaccio (armando-migliaccio)]

This is an automated cleanup. This bug report has been closed because it
is older than 18 months and there is no open code change to fix this.
After this time it is unlikely that the circumstances which lead to
the observed issue can be reproduced.

If you can reproduce the bug, please:
* reopen the bug report (set to status "New")
* AND add the detailed steps to reproduce the issue (if applicable)
* AND leave a comment "CONFIRMED FOR: <RELEASE_NAME>"
  Only still supported release names are valid (INCUBATOR-JUNO, LIBERTY, MITAKA, NEWTON).
  Valid example: CONFIRMED FOR: INCUBATOR-JUNO