Ubuntu
neutron-fwaas package

Neutron db sync fails

Bug #1863257 reported by TAHARDJEBBAR Abdellah Seddik on 2020-02-14

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	neutron-fwaas (Ubuntu)	Expired	Medium	Unassigned

Bug Description

When trying to sync db with python3-neutron-fwaas, db sync fails due to this code:

    op.create_primary_key(
        'pk_firewall_group_port_associations_v2',
        'firewall_group_port_associations_v2',
        ['firewall_group_id', 'port_id'])

when commenting the code the db sync the db sync works, the change is described in:
https://launchpadlibrarian.net/453954118/neutron-fwaas_1%3A14.0.0-0ubuntu1.1_1%3A14.0.0-0ubuntu1.2.diff.gz

DB sync stack:

neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade heads
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
  Running upgrade for neutron ...
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade -> kilo
INFO [alembic.runtime.migration] Running upgrade kilo -> 354db87e3225
INFO [alembic.runtime.migration] Running upgrade 354db87e3225 -> 599c6a226151
INFO [alembic.runtime.migration] Running upgrade 599c6a226151 -> 52c5312f6baf
INFO [alembic.runtime.migration] Running upgrade 52c5312f6baf -> 313373c0ffee
INFO [alembic.runtime.migration] Running upgrade 313373c0ffee -> 8675309a5c4f
INFO [alembic.runtime.migration] Running upgrade 8675309a5c4f -> 45f955889773
INFO [alembic.runtime.migration] Running upgrade 45f955889773 -> 26c371498592
INFO [alembic.runtime.migration] Running upgrade 26c371498592 -> 1c844d1677f7
INFO [alembic.runtime.migration] Running upgrade 1c844d1677f7 -> 1b4c6e320f79
INFO [alembic.runtime.migration] Running upgrade 1b4c6e320f79 -> 48153cb5f051
INFO [alembic.runtime.migration] Running upgrade 48153cb5f051 -> 9859ac9c136
INFO [alembic.runtime.migration] Running upgrade 9859ac9c136 -> 34af2b5c5a59
INFO [alembic.runtime.migration] Running upgrade 34af2b5c5a59 -> 59cb5b6cf4d
INFO [alembic.runtime.migration] Running upgrade 59cb5b6cf4d -> 13cfb89f881a
INFO [alembic.runtime.migration] Running upgrade 13cfb89f881a -> 32e5974ada25
INFO [alembic.runtime.migration] Running upgrade 32e5974ada25 -> ec7fcfbf72ee
INFO [alembic.runtime.migration] Running upgrade ec7fcfbf72ee -> dce3ec7a25c9
INFO [alembic.runtime.migration] Running upgrade dce3ec7a25c9 -> c3a73f615e4
INFO [alembic.runtime.migration] Running upgrade c3a73f615e4 -> 659bf3d90664
INFO [alembic.runtime.migration] Running upgrade 659bf3d90664 -> 1df244e556f5
INFO [alembic.runtime.migration] Running upgrade 1df244e556f5 -> 19f26505c74f
INFO [alembic.runtime.migration] Running upgrade 19f26505c74f -> 15be73214821
INFO [alembic.runtime.migration] Running upgrade 15be73214821 -> b4caf27aae4
INFO [alembic.runtime.migration] Running upgrade b4caf27aae4 -> 15e43b934f81
INFO [alembic.runtime.migration] Running upgrade 15e43b934f81 -> 31ed664953e6
INFO [alembic.runtime.migration] Running upgrade 31ed664953e6 -> 2f9e956e7532
INFO [alembic.runtime.migration] Running upgrade 2f9e956e7532 -> 3894bccad37f
INFO [alembic.runtime.migration] Running upgrade 3894bccad37f -> 0e66c5227a8a
INFO [alembic.runtime.migration] Running upgrade 0e66c5227a8a -> 45f8dd33480b
INFO [alembic.runtime.migration] Running upgrade 45f8dd33480b -> 5abc0278ca73
INFO [alembic.runtime.migration] Running upgrade 5abc0278ca73 -> d3435b514502
INFO [alembic.runtime.migration] Running upgrade d3435b514502 -> 30107ab6a3ee
INFO [alembic.runtime.migration] Running upgrade 30107ab6a3ee -> c415aab1c048
INFO [alembic.runtime.migration] Running upgrade c415aab1c048 -> a963b38d82f4
INFO [alembic.runtime.migration] Running upgrade kilo -> 30018084ec99
INFO [alembic.runtime.migration] Running upgrade 30018084ec99 -> 4ffceebfada
INFO [alembic.runtime.migration] Running upgrade 4ffceebfada -> 5498d17be016
INFO [alembic.runtime.migration] Running upgrade 5498d17be016 -> 2a16083502f3
INFO [alembic.runtime.migration] Running upgrade 2a16083502f3 -> 2e5352a0ad4d
INFO [alembic.runtime.migration] Running upgrade 2e5352a0ad4d -> 11926bcfe72d
INFO [alembic.runtime.migration] Running upgrade 11926bcfe72d -> 4af11ca47297
INFO [alembic.runtime.migration] Running upgrade 4af11ca47297 -> 1b294093239c
INFO [alembic.runtime.migration] Running upgrade 1b294093239c -> 8a6d8bdae39
INFO [alembic.runtime.migration] Running upgrade 8a6d8bdae39 -> 2b4c2465d44b
INFO [alembic.runtime.migration] Running upgrade 2b4c2465d44b -> e3278ee65050
INFO [alembic.runtime.migration] Running upgrade e3278ee65050 -> c6c112992c9
INFO [alembic.runtime.migration] Running upgrade c6c112992c9 -> 5ffceebfada
INFO [alembic.runtime.migration] Running upgrade 5ffceebfada -> 4ffceebfcdc
INFO [alembic.runtime.migration] Running upgrade 4ffceebfcdc -> 7bbb25278f53
INFO [alembic.runtime.migration] Running upgrade 7bbb25278f53 -> 89ab9a816d70
INFO [alembic.runtime.migration] Running upgrade 89ab9a816d70 -> c879c5e1ee90
INFO [alembic.runtime.migration] Running upgrade c879c5e1ee90 -> 8fd3918ef6f4
INFO [alembic.runtime.migration] Running upgrade 8fd3918ef6f4 -> 4bcd4df1f426
INFO [alembic.runtime.migration] Running upgrade 4bcd4df1f426 -> b67e765a3524
INFO [alembic.runtime.migration] Running upgrade a963b38d82f4 -> 3d0e74aa7d37
INFO [alembic.runtime.migration] Running upgrade 3d0e74aa7d37 -> 030a959ceafa
INFO [alembic.runtime.migration] Running upgrade 030a959ceafa -> a5648cfeeadf
INFO [alembic.runtime.migration] Running upgrade a5648cfeeadf -> 0f5bef0f87d4
INFO [alembic.runtime.migration] Running upgrade 0f5bef0f87d4 -> 67daae611b6e
INFO [alembic.runtime.migration] Running upgrade b67e765a3524 -> a84ccf28f06a
INFO [alembic.runtime.migration] Running upgrade a84ccf28f06a -> 7d9d8eeec6ad
INFO [alembic.runtime.migration] Running upgrade 7d9d8eeec6ad -> a8b517cff8ab
INFO [alembic.runtime.migration] Running upgrade a8b517cff8ab -> 3b935b28e7a0
INFO [alembic.runtime.migration] Running upgrade 3b935b28e7a0 -> b12a3ef66e62
INFO [alembic.runtime.migration] Running upgrade b12a3ef66e62 -> 97c25b0d2353
INFO [alembic.runtime.migration] Running upgrade 97c25b0d2353 -> 2e0d7a8a1586
INFO [alembic.runtime.migration] Running upgrade 2e0d7a8a1586 -> 5c85685d616d
INFO [alembic.runtime.migration] Running upgrade 5c85685d616d -> 33e74c25a5b9
INFO [alembic.runtime.migration] Running upgrade 67daae611b6e -> 6b461a21bcfc
INFO [alembic.runtime.migration] Running upgrade 6b461a21bcfc -> 5cd92597d11d
INFO [alembic.runtime.migration] Running upgrade 5cd92597d11d -> 929c968efe70
INFO [alembic.runtime.migration] Running upgrade 929c968efe70 -> a9c43481023c
INFO [alembic.runtime.migration] Running upgrade a9c43481023c -> 804a3c76314c
INFO [alembic.runtime.migration] Running upgrade 804a3c76314c -> 2b42d90729da
INFO [alembic.runtime.migration] Running upgrade 2b42d90729da -> 62c781cb6192
INFO [alembic.runtime.migration] Running upgrade 62c781cb6192 -> c8c222d42aa9
INFO [alembic.runtime.migration] Running upgrade c8c222d42aa9 -> 349b6fd605a6
INFO [alembic.runtime.migration] Running upgrade 349b6fd605a6 -> 7d32f979895f
INFO [alembic.runtime.migration] Running upgrade 7d32f979895f -> 594422d373ee
INFO [alembic.runtime.migration] Running upgrade 594422d373ee -> 61663558142c
INFO [alembic.runtime.migration] Running upgrade 61663558142c -> 867d39095bf4, port forwarding
INFO [alembic.runtime.migration] Running upgrade 867d39095bf4 -> d72db3e25539, modify uniq port forwarding
INFO [alembic.runtime.migration] Running upgrade d72db3e25539 -> cada2437bf41
INFO [alembic.runtime.migration] Running upgrade cada2437bf41 -> 195176fb410d, router gateway IP QoS
INFO [alembic.runtime.migration] Running upgrade 195176fb410d -> fb0167bd9639
INFO [alembic.runtime.migration] Running upgrade fb0167bd9639 -> 0ff9e3881597
INFO [alembic.runtime.migration] Running upgrade 0ff9e3881597 -> 9bfad3f1e780
INFO [alembic.runtime.migration] Running upgrade 9bfad3f1e780 -> 68f83c3e007f
  OK
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
  Running upgrade for neutron-fwaas ...
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade -> start_neutron_fwaas, start neutron-fwaas chain
INFO [alembic.runtime.migration] Running upgrade start_neutron_fwaas -> 4202e3047e47, add_index_tenant_id
INFO [alembic.runtime.migration] Running upgrade 4202e3047e47 -> 540142f314f4, FWaaS router insertion
INFO [alembic.runtime.migration] Running upgrade 540142f314f4 -> 796c68dffbb, cisco_csr_fwaas
INFO [alembic.runtime.migration] Running upgrade 796c68dffbb -> kilo, kilo
INFO [alembic.runtime.migration] Running upgrade kilo -> c40fbb377ad, Initial Liberty no-op script.
INFO [alembic.runtime.migration] Running upgrade c40fbb377ad -> 4b47ea298795, add reject rule
INFO [alembic.runtime.migration] Running upgrade 4b47ea298795 -> d6a12e637e28, neutron-fwaas v2.0
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 588, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/lib/python3/dist-packages/pymysql/cursors.py", line 165, in execute
    result = self._query(query)
  File "/usr/lib/python3/dist-packages/pymysql/cursors.py", line 321, in _query
    conn.query(q)
  File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 860, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 1061, in _read_query_result
    result.read()
  File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 1349, in read
    first_packet = self.connection._read_packet()
  File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 1018, in _read_packet
    packet.check_error()
  File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 384, in check_error
    err.raise_mysql_exception(self._data)
  File "/usr/lib/python3/dist-packages/pymysql/err.py", line 107, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.InternalError: (1832, "Cannot change column 'firewall_group_id': used in a foreign key constraint 'firewall_group_port_associations_v2_ibfk_1'")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/bin/neutron-db-manage", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python3/dist-packages/neutron/db/migration/cli.py", line 656, in main
    return_val |= bool(CONF.command.func(config, CONF.command.name))
  File "/usr/lib/python3/dist-packages/neutron/db/migration/cli.py", line 180, in do_upgrade
    desc=branch, sql=CONF.command.sql)
  File "/usr/lib/python3/dist-packages/neutron/db/migration/cli.py", line 81, in do_alembic_command
    getattr(alembic_command, cmd)(config, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/alembic/command.py", line 298, in upgrade
    script.run_env()
  File "/usr/local/lib/python3.6/dist-packages/alembic/script/base.py", line 489, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/usr/local/lib/python3.6/dist-packages/alembic/util/pyfiles.py", line 98, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/local/lib/python3.6/dist-packages/alembic/util/compat.py", line 173, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/lib/python3/dist-packages/neutron_fwaas/db/migration/alembic_migrations/env.py", line 86, in <module>
    run_migrations_online()
  File "/usr/lib/python3/dist-packages/neutron_fwaas/db/migration/alembic_migrations/env.py", line 77, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/local/lib/python3.6/dist-packages/alembic/runtime/environment.py", line 846, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/lib/python3.6/dist-packages/alembic/runtime/migration.py", line 518, in run_migrations
    step.migration_fn(**kw)
  File "/usr/lib/python3/dist-packages/neutron_fwaas/db/migration/alembic_migrations/versions/newton/expand/d6a12e637e28_neutron_fwaas_v2_0.py", line 108, in upgrade
    ['firewall_group_id', 'port_id'])
  File "<string>", line 8, in create_primary_key
  File "<string>", line 3, in create_primary_key
  File "/usr/local/lib/python3.6/dist-packages/alembic/operations/ops.py", line 290, in create_primary_key
    return operations.invoke(op)
  File "/usr/local/lib/python3.6/dist-packages/alembic/operations/base.py", line 374, in invoke
    return fn(self, operation)
  File "/usr/local/lib/python3.6/dist-packages/alembic/operations/toimpl.py", line 151, in create_constraint
    operation.to_constraint(operations.migration_context)
  File "/usr/local/lib/python3.6/dist-packages/alembic/ddl/impl.py", line 244, in add_constraint
    self._exec(schema.AddConstraint(const))
  File "/usr/local/lib/python3.6/dist-packages/alembic/ddl/impl.py", line 140, in _exec
    return conn.execute(construct, *multiparams, **params)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 982, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1044, in _execute_ddl
    compiled,
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1250, in _execute_context
    e, statement, parameters, cursor, context
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1474, in _handle_dbapi_exception
    util.raise_from_cause(newraise, exc_info)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 152, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 588, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/lib/python3/dist-packages/pymysql/cursors.py", line 165, in execute
    result = self._query(query)
  File "/usr/lib/python3/dist-packages/pymysql/cursors.py", line 321, in _query
    conn.query(q)
  File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 860, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 1061, in _read_query_result
    result.read()
  File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 1349, in read
    first_packet = self.connection._read_packet()
  File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 1018, in _read_packet
    packet.check_error()
  File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 384, in check_error
    err.raise_mysql_exception(self._data)
  File "/usr/lib/python3/dist-packages/pymysql/err.py", line 107, in raise_mysql_exception
    raise errorclass(errno, errval)
oslo_db.exception.DBError: (pymysql.err.InternalError) (1832, "Cannot change column 'firewall_group_id': used in a foreign key constraint 'firewall_group_port_associations_v2_ibfk_1'")
[SQL: ALTER TABLE firewall_group_port_associations_v2 ADD CONSTRAINT pk_firewall_group_port_associations_v2 PRIMARY KEY (firewall_group_id, port_id)]
(Background on this error at: http://sqlalche.me/e/2j85)

Package Info:

Package: python3-neutron-fwaas
Version: 1:14.0.1-0ubuntu1~cloud0
Priority: optional
Section: python
Source: neutron-fwaas
Maintainer: Ubuntu Developers <email address hidden>
Installed-Size: 1195 kB
Depends: neutron-fwaas-common (= 1:14.0.1-0ubuntu1~cloud0), python3-alembic (>= 0.8.10), python3-eventlet (>= 0.18.2), python3-netaddr (>= 0.7.18), python3-neutron (>= 2:14.0.0~), python3-neutron-lib (>= 1.25.0), python3-os-ken (>= 0.3.0), python3-oslo.config (>= 1:5.2.0), python3-oslo.db (>= 4.37.0), python3-oslo.log (>= 3.36.0), python3-oslo.messaging (>= 5.29.0), python3-oslo.privsep (>= 1.32.0), python3-oslo.serialization (>= 2.18.0), python3-oslo.service (>= 1.24.0), python3-oslo.utils (>= 3.33.0), python3-pbr (>= 2.0.0), python3-pyroute2 (>= 0.4.21), python3-six (>= 1.10.0), python3-sqlalchemy (>= 1.2.0), python3:any
Suggests: python3-zmq
Supported: 48m
Download-Size: 138 kB
APT-Manual-Installed: no
APT-Sources: http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/stein/main amd64 Packages
Description: Firewall-as-a-Service driver for OpenStack Neutron

Db info:

Package: mariadb-server
Version: 1:10.1.44+maria-1~bionic
Priority: optional
Section: database
Source: mariadb-10.1
Maintainer: MariaDB Developers <email address hidden>
Installed-Size: 10.2 kB
Depends: mariadb-server-10.1 (= 1:10.1.44+maria-1~bionic)
Homepage: http://mariadb.org/
Download-Size: 3012 B
APT-Manual-Installed: yes
APT-Sources: http://lon1.mirrors.digitalocean.com/mariadb/repo/10.1/ubuntu bionic/main amd64 Packages

Revision history for this message

James Page (james-page) wrote on 2020-03-24:

That primary key was added to deal with bug 1831986 and also appears to work fine with MySQL8 (in addition to PXC 5.7.x)

I suspect that this is something todo with the mode that your have MariaDB operating in but I'm not 100% sure.

Changed in neutron-fwaas (Ubuntu):
status:	New → Incomplete
importance:	Undecided → Medium

Revision history for this message

Launchpad Janitor (janitor) wrote on 2020-05-24:

[Expired for neutron-fwaas (Ubuntu) because there has been no activity for 60 days.]

Changed in neutron-fwaas (Ubuntu):
status:	Incomplete → Expired

Revision history for this message

Sagar Gurung (sagargurung) wrote on 2020-07-17:

Got same errors.
Solved by commenting out below lines of code under this (d6a12e637e28_neutron_fwaas_v2_0.py) python file:

op.create_primary_key(
        'pk_firewall_group_port_associations_v2',
        'firewall_group_port_associations_v2',
        ['firewall_group_id', 'port_id'])

For anyone new encountering same issue, follow tracebacks thoroughly and you shall see something like this:

File "/usr/lib/python3/dist-packages/neutron_fwaas/db/migration/alembic_migrations/versions/newton/expand/d6a12e637e28_neutron_fwaas_v2_0.py", line 107, in upgrade

to know your similar file name. CD into that dir and vim it, and comment out above lines of code that says op.create_primary_key ...

Once, you make those changes, you also need to delete this table (firewall_policies_v2) which is under neutron database. However, you won't be able to delete it as it has foreign key connected to some other tables. Solution to this is to rather delete entire neutron database (DROP database neutron).

After that, re-create neutron database and just apply:

su -s /bin/sh -c "neutron-db-manage --config-file /etc/neutron/neutron.conf \
--config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade head" neutron

to populate Neutron database.

Hope this helps.

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.

Ubuntuneutron-fwaas package

Neutron db sync fails

Bug Description

Other bug subscribers

Remote bug watches

Ubuntu
neutron-fwaas package