agent model migration script does not contain ml2 plugin

[yong sheng gong]

+ mysql -uroot -proot -h127.0.0.1 -e 'DROP DATABASE IF EXISTS neutron_ml2;'
+ mysql -uroot -proot -h127.0.0.1 -e 'CREATE DATABASE neutron_ml2 CHARACTER SET utf8;'
+ /usr/local/bin/neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head
No handlers could be found for logger "neutron.common.legacy"
INFO [alembic.migration] Context impl MySQLImpl.
INFO [alembic.migration] Will assume non-transactional DDL.
INFO [alembic.migration] Running upgrade None -> folsom, folsom initial database
INFO [alembic.migration] Running upgrade folsom -> 2c4af419145b, l3_support
INFO [alembic.migration] Running upgrade 2c4af419145b -> 5a875d0e5c, ryu
INFO [alembic.migration] Running upgrade 5a875d0e5c -> 48b6f43f7471, DB support for service types
INFO [alembic.migration] Running upgrade 48b6f43f7471 -> 3cb5d900c5de, security_groups
INFO [alembic.migration] Running upgrade 3cb5d900c5de -> 1d76643bcec4, nvp_netbinding
INFO [alembic.migration] Running upgrade 1d76643bcec4 -> 2a6d0b51f4bb, cisco plugin cleanup
INFO [alembic.migration] Running upgrade 2a6d0b51f4bb -> 1b693c095aa3, Quota ext support added in Grizzly
INFO [alembic.migration] Running upgrade 1b693c095aa3 -> 1149d7de0cfa, inital port security
INFO [alembic.migration] Running upgrade 1149d7de0cfa -> 49332180ca96, ryu plugin update
INFO [alembic.migration] Running upgrade 49332180ca96 -> 38335592a0dc, nvp_portmap
INFO [alembic.migration] Running upgrade 38335592a0dc -> 54c2c487e913, 'DB support for load balancing service
INFO [alembic.migration] Running upgrade 54c2c487e913 -> 45680af419f9, nvp_qos
INFO [alembic.migration] Running upgrade 45680af419f9 -> 1c33fa3cd1a1, Support routing table configuration on Router
INFO [alembic.migration] Running upgrade 1c33fa3cd1a1 -> 363468ac592c, nvp_network_gw
INFO [alembic.migration] Running upgrade 363468ac592c -> 511471cc46b, Add agent management extension model support
INFO [alembic.migration] Running upgrade 511471cc46b -> 3b54bf9e29f7, NEC plugin sharednet
INFO [alembic.migration] Running upgrade 3b54bf9e29f7 -> 4692d074d587, agent scheduler
INFO [alembic.migration] Running upgrade 4692d074d587 -> 1341ed32cc1e, nvp_net_binding
INFO [alembic.migration] Running upgrade 1341ed32cc1e -> grizzly, grizzly
INFO [alembic.migration] Running upgrade grizzly -> f489cf14a79c, DB support for load balancing service (havana)
INFO [alembic.migration] Running upgrade f489cf14a79c -> 176a85fc7d79, Add portbindings db
INFO [alembic.migration] Running upgrade 176a85fc7d79 -> 32b517556ec9, remove TunnelIP model
INFO [alembic.migration] Running upgrade 32b517556ec9 -> 128e042a2b68, ext_gw_mode
INFO [alembic.migration] Running upgrade 128e042a2b68 -> 5ac71e65402c, ml2_initial
INFO [alembic.migration] Running upgrade 5ac71e65402c -> 3cbf70257c28, nvp_mac_learning
INFO [alembic.migration] Running upgrade 3cbf70257c28 -> 5918cbddab04, add tables for router rules support
INFO [alembic.migration] Running upgrade 5918cbddab04 -> 3cabb850f4a5, Table to track port to host associations
INFO [alembic.migration] Running upgrade 3cabb850f4a5 -> b7a8863760e, Remove cisco_vlan_bindings table
INFO [alembic.migration] Running upgrade b7a8863760e -> 13de305df56e, nec_add_pf_name
INFO [alembic.migration] Running upgrade 13de305df56e -> 20ae61555e95, DB Migration for ML2 GRE Type Driver
INFO [alembic.migration] Running upgrade 20ae61555e95 -> 477a4488d3f4, DB Migration for ML2 VXLAN Type Driver
INFO [alembic.migration] Running upgrade 477a4488d3f4 -> 2032abe8edac, LBaaS add status description
INFO [alembic.migration] Running upgrade 2032abe8edac -> 52c5e4a18807, LBaaS Pool scheduler
INFO [alembic.migration] Running upgrade 52c5e4a18807 -> 557edfc53098, New service types framework (service providers)
INFO [alembic.migration] Running upgrade 557edfc53098 -> e6b16a30d97, Add cisco_provider_networks table
INFO [alembic.migration] Running upgrade e6b16a30d97 -> 39cf3f799352, FWaaS Havana-2 model
INFO [alembic.migration] Running upgrade 39cf3f799352 -> 52ff27f7567a, Support for VPNaaS
INFO [alembic.migration] Running upgrade 52ff27f7567a -> 11c6e18605c8, Pool Monitor status field
INFO [alembic.migration] Running upgrade 11c6e18605c8 -> 35c7c198ddea, remove status from HealthMonitor
INFO [alembic.migration] Running upgrade 35c7c198ddea -> 263772d65691, Cisco plugin db cleanup part II
INFO [alembic.migration] Running upgrade 263772d65691 -> c88b6b5fea3, Cisco N1KV tables
INFO [alembic.migration] Running upgrade c88b6b5fea3 -> f9263d6df56, remove_dhcp_lease
INFO [alembic.migration] Running upgrade f9263d6df56 -> 569e98a8132b, metering
INFO [alembic.migration] Running upgrade 569e98a8132b -> 86cf4d88bd3, remove bigswitch port tracking table
INFO [alembic.migration] Running upgrade 86cf4d88bd3 -> 3c6e57a23db4, add multiprovider
INFO [alembic.migration] Running upgrade 3c6e57a23db4 -> 63afba73813, Add unique constraint for id column of TunnelEndpoint
INFO [alembic.migration] Running upgrade 63afba73813 -> 40dffbf4b549, nvp_dist_router
INFO [alembic.migration] Running upgrade 40dffbf4b549 -> 53bbd27ec841, Extra dhcp opts support
INFO [alembic.migration] Running upgrade 53bbd27ec841 -> 46a0efbd8f0, cisco_n1kv_multisegment_trunk
INFO [alembic.migration] Running upgrade 46a0efbd8f0 -> 2a3bae1ceb8, NEC Port Binding
INFO [alembic.migration] Running upgrade 2a3bae1ceb8 -> 14f24494ca31, DB Migration for Arista ml2 mechanism driver
INFO [alembic.migration] Running upgrade 14f24494ca31 -> 32a65f71af51, ml2 portbinding
INFO [alembic.migration] Running upgrade 32a65f71af51 -> 66a59a7f516, NEC OpenFlow Router
INFO [alembic.migration] Running upgrade 66a59a7f516 -> 51b4de912379, Cisco Nexus ML2 mechanism driver
INFO [alembic.migration] Running upgrade 51b4de912379 -> 1efb85914233, allowedaddresspairs
INFO [alembic.migration] Running upgrade 1efb85914233 -> 38fc1f6789f8, Cisco N1KV overlay support
INFO [alembic.migration] Running upgrade 38fc1f6789f8 -> 4a666eb208c2, service router
INFO [alembic.migration] Running upgrade 4a666eb208c2 -> 338d7508968c, vpnaas peer_address size increase
INFO [alembic.migration] Running upgrade 338d7508968c -> 3ed8f075e38a, nvp fwaas plugin
INFO [alembic.migration] Running upgrade 3ed8f075e38a -> 3d6fae8b70b0, nvp lbaas plugin
INFO [alembic.migration] Running upgrade 3d6fae8b70b0 -> 1064e98b7917, nec-pf-port-del
INFO [alembic.migration] Running upgrade 1064e98b7917 -> 2528ceb28230, NEC PacketFilter network_id nullable fix
INFO [alembic.migration] Running upgrade 2528ceb28230 -> 3a520dd165d0, Cisco Nexus multi-switch
INFO [alembic.migration] Running upgrade 3a520dd165d0 -> 27ef74513d33, quota_in_plumgrid_plugin
INFO [alembic.migration] Running upgrade 27ef74513d33 -> havana, havana
INFO [alembic.migration] Running upgrade havana -> e197124d4b9, add unique constraint to members
INFO [alembic.migration] Running upgrade e197124d4b9 -> 1fcfc149aca4, Add a unique constraint on (agent_type, host) columns to prevent a race
condition when an agent entry is 'upserted'.
Traceback (most recent call last):
  File "/usr/local/bin/neutron-db-manage", line 10, in <module>
    sys.exit(main())
  File "/mnt/data/opt/stack/neutron/neutron/db/migration/cli.py", line 143, in main
    CONF.command.func(config, CONF.command.name)
  File "/mnt/data/opt/stack/neutron/neutron/db/migration/cli.py", line 80, in do_upgrade_downgrade
    do_alembic_command(config, cmd, revision, sql=CONF.command.sql)
  File "/mnt/data/opt/stack/neutron/neutron/db/migration/cli.py", line 59, in do_alembic_command
    getattr(alembic_command, cmd)(config, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/alembic/command.py", line 124, in upgrade
    script.run_env()
  File "/usr/local/lib/python2.7/dist-packages/alembic/script.py", line 193, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/usr/local/lib/python2.7/dist-packages/alembic/util.py", line 177, in load_python_file
    module = load_module(module_id, path)
  File "/usr/local/lib/python2.7/dist-packages/alembic/compat.py", line 39, in load_module
    return imp.load_source(module_id, path, fp)
  File "/mnt/data/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 105, in <module>
    run_migrations_online()
  File "/mnt/data/opt/stack/neutron/neutron/db/migration/alembic_migrations/env.py", line 89, in run_migrations_online
    options=build_options())
  File "<string>", line 7, in run_migrations
  File "/usr/local/lib/python2.7/dist-packages/alembic/environment.py", line 652, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/lib/python2.7/dist-packages/alembic/migration.py", line 224, in run_migrations
    change(**kw)
  File "/mnt/data/opt/stack/neutron/neutron/db/migration/alembic_migrations/versions/1fcfc149aca4_agents_unique_by_type_and_host.py", line 50, in upgrade
    local_cols=['agent_type', 'host']
  File "<string>", line 7, in create_unique_constraint
  File "/usr/local/lib/python2.7/dist-packages/alembic/operations.py", line 539, in create_unique_constraint
    schema=schema, **kw)
  File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 135, in add_constraint
    self._exec(schema.AddConstraint(const))
  File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 76, in _exec
    conn.execute(construct, *multiparams, **params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1449, in execute
    params)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1542, in _execute_ddl
    compiled
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1698, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1691, in _execute_context
    context)
  File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 331, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
sqlalchemy.exc.ProgrammingError: (ProgrammingError) (1146, "Table 'neutron_ml2.agents' doesn't exist") 'ALTER TABLE agents ADD CONSTRAINT uniq_agents0agent_type0host UNIQUE (agent_type, host)' ()
++ failed
++ local r=1
+++ jobs -p
++ kill
++ set +o xtrace

[yong sheng gong (gongysh)]

I am running:

cd /opt/stack/neutron
git checkout master
git pull
cd ~/devstack
OFFLINE=True ./stack.sh

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/61677

[yong sheng gong (gongysh)]

my localrc:
disable_service n-net cinder c-sch c-api c-vol
KEYSTONE_TOKEN_FORMAT=UUID
LOG_COLOR=False
HOST_IP=172.16.108.1
#VNCSERVER_LISTEN=0.0.0.0
#DEST=/opt/stack1
#VNCSERVER_PROXYCLIENT_ADDRESS=$HOST_IP
enable_service q-svc
#VNCSERVER_LISTEN=$SERVICE_HOST
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service neutron
#enable_service q-lbaas
#enable_service q-fwaas
#enable_service q-vpn
Q_PLUGIN=ml2
Q_AGENT=openvswitch
Q_AGENT_EXTRA_AGENT_OPTS=(l2_population=True)
Q_ML2_TENANT_NETWORK_TYPE=(gre vxlan)
Q_ML2_PLUGIN_MECHANISM_DRIVERS=openvswitch,linuxbridge,l2population
ENABLE_TENANT_VLANS=True
ML2_VLAN_RANGES=physnet1:1000:2999
OVS_BRIDGE_MAPPINGS=physnet1:br-eth0
DATABASE_PASSWORD=root
RABBIT_PASSWORD=password
SERVICE_TOKEN=token
SERVICE_PASSWORD=password
ADMIN_PASSWORD=password
SCREEN_LOGDIR=$DEST/logs/screen
PUBLIC_NETWORK_NAME=office_net
FLOATING_RANGE=192.168.30.0/24
PUBLIC_NETWORK_GATEWAY=192.168.30.1
Q_FLOATING_ALLOCATION_POOL=start=192.168.30.10,end=192.168.30.40

[Eugene Nikanorov (enikanorov)]

I believe this should also be backported

[Ben Nemec (bnemec)]

Eugene, this migration does not exist in Havana, does it? I thought this was a relatively recent change in icehouse.

For anyone who is being blocked on using devstack with Neutron because of this, my workaround was to set Q_PLUGIN=openvswitch in my localrc. That allows devstack to complete successfully and Neutron seems to be working fine using openvswitch directly instead of through ml2.

[Eugene Nikanorov (enikanorov)]

Ben, another option for devstack would be to enable q-lbaas service. Loadbalancer is one of the plugins for which agents table is created. So in this case q-lbaas service is enabled, migrations should work fine.

[Henry Gessau (gessau)]

The broken migrations for ML2 were discussed in the neutron meeting:
http://eavesdrop.openstack.org/meetings/neutron/2013/neutron.2013-11-25-21.01.log.html

and follow-up discussions took place on #openstack-neutron:
http://eavesdrop.openstack.org/irclogs/%23openstack-neutron/%23openstack-neutron.2013-12-23.log
See time-stamps 2013-12-23T22:01:45 to 2013-12-23T22:36:29

[dkehn (dekehn)]

This address this issues https://review.openstack.org/#/c/61663/

[Benoit Sigoure (tsunanet)]

FWIW, enabling q-lbaas worked for me, but patching in the change up for review didn't. I'm using a fresh install of devstack master @ 6fbb28d0.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/79660
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=73cbbd0809d52f969ba82e85be32749dd4c17d25
Submitter: Jenkins
Branch: master

commit 73cbbd0809d52f969ba82e85be32749dd4c17d25
Author: Jakub Libosvar <email address hidden>
Date: Tue Mar 11 17:11:38 2014 +0100

    Create agents table when ML2 core_plugin is used

    In Icehouse agents table is not created when ml2 is used but later in
    migration chain is added uniq constraint on this table. This patch makes
    agents table created when ml2 plugin is used as core_plugin.

    Closes-bug: 1260224
    Change-Id: I501534261efd278b6a69a66758f1a01aa49bfa17