Long name of the network causes nailgun freeze

[Max Lvov]

When you are trying to create new network in fuel and then deploy env, nailgun stuck with 100% cpu load if network name was 12+ letters (did not check exact border for length but it fails for 12 and works for everything shorter then "management")

Environment MOS 8.0 MU1
3 Controllers, 3 mongo, 4 computes, 4 ceph

Steps to reproduce:
1 On fuel master, add new network by executing command fuel network-group --create --name <name_longer_then_10_letters>
2 Go to web interface, clear checkbox "use whole CIDR" and set some range. (Probably bug will appear even without this step)
3 Press deploy

Expected:
Deployment starts

What I see:
Web interface for this particular env stops to open
If execute "top" on fuel master uwsgi process consumes 100% CPU

Workaround:
Use network name shorter then "management"

How to fix crush:
dockerctl reload nailgun
fuel task
fuel task --force --delete <id of "deploy" task>

[Ilya Kharin (akscram)]

@Max, could you provide some description details of your environment, at least the version? Also, please follow the guide how to report a bug [1], there there is a template how to provide fully informative description.

[1] https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Here_is_how_you_file_a_bug

[Dmitry Dmitriev (dmdmitriev)]

Can not reproduce the problem on two virtual nodes cloud under Fuel 8 (without MU).

Network configuration is:

[root@nailgun ~]# fuel network-group --env=1 --list
id | name | vlan_start | cidr | gateway | group_id
---|---------------------|------------|----------------|-------------|---------
5 | private | None | None | None | 1
11 | verylongnamenetwork | None | 172.16.1.0/24 | None | 1
1 | fuelweb_admin | None | 10.109.15.0/24 | 10.109.15.2 | None
2 | public | None | 10.109.17.0/24 | 10.109.17.1 | 1
4 | storage | None | 10.109.16.0/24 | None | 1
3 | management | None | 10.109.18.0/24 | None | 1
[root@nailgun ~]#

[Dmitry Dmitriev (dmdmitriev)]

Max Lvov:

Exact names of custom networks in customer cloud:
prod-internal
non-prod-internal
prod-external
non-prod-external

(4 networks)

[Dmitry Dmitriev (dmdmitriev)]

Max Lvov:

Network template was used and it will be attached to the bug.

[Max Lvov (usrleon)]

Network template attached

[Dmitry Dmitriev (dmdmitriev)]

I have adopted uploaded template to my virtual environment and successfully deployed the cloud with original network names:

prod-internal
non-prod-internal
prod-external
non-prod-external

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/390118
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=70fad4145db2446216cdac02985355b6c410e0a9
Submitter: Jenkins
Branch: master

commit 70fad4145db2446216cdac02985355b6c410e0a9
Author: Evgeny L <email address hidden>
Date: Wed Oct 19 13:29:00 2016 +0000

    Workaround is required to prevent deadlock in task serializers.

    Due to a bug in core python library [1], if iterator passed
    to imap or imap_unordered raises an exception, it hangs in
    deadlock forever, the issue is fixed in python 2.7.10, but
    upstream repositories have much older version.

    [1] http://bugs.python.org/issue23051

    Closes-Bug: #1602365
    Change-Id: I0676de8f74cf4e0d248e0a074b58960bf86b7fbd

[Alexander Kislitsky (akislitsky)]

Fix merged to stable/mitaka: https://review.openstack.org/#/c/388706/

[Alexander Kislitsky (akislitsky)]

This bug doesn't affect Fuel 8.0 - we doesn't use multiprocessing.Pool. Marking as invalid for the Fuel 8.0 updates.

[Max Lvov (usrleon)]

That's weird, I got this effect on 8.0 MU3

[Alexey. Kalashnikov (akalashnikov)]

verified on 9.2 snapshot #465

shotgun2 report:
http://paste.openstack.org/show/589041/

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/fuel-web 11.0.0.0rc1 release candidate.