[library] Moving of MySQL network role away from management network causes deployment failure

[Artem Panchenko]

Fuel version info (7.0 custom iso #675): http://paste.openstack.org/show/406032/

Environment deployment with network template fails if 'mgmt/database' role is moved to storage network:

node-1 2015-07-28T07:17:27.521578 err: (/Stage[main]/Keystone::Db::Mysql/Openstacklib::Db::Mysql[keystone]/Openstacklib::Db::Mysql::Host_access[keystone_localhost]/Database_user[keystone@localhost]/ensure) change from absent to present failed: Execution of '/usr/bin/mysql --defaults-extra-file=/root/.my.cnf mysql -e create user 'keystone'@'localhost' identified by PASSWORD '*DC95241B069650FBFBC3C90AC76F63ABA8BBB2DF'' returned 1: ERROR 1045 (28000): Access denied for user 'root'@'240.0.0.2' (using password: YES)

Here is network template I used:

http://paste.openstack.org/show/406033/

Steps to reproduce:

1. Create environment: neutron+VLAN, Ceph for all
2. Add 3 controller+ceph and 2 compute+ceph nodes
3. Upload network template (provided above)
4. Deploy changes

Result:

 - deployment failed, because user 'kesytone' for MySQL couldn't be created

The issue is caused by incorrect 'root' user settings for MySQL:

root@node-1:~# ip netns exec haproxy /usr/bin/mysql --defaults-extra-file=/tmp/.my.cnf mysql
ERROR 1045 (28000): Access denied for user 'root'@'240.0.0.2' (using password: YES)
root@node-1:~# /usr/bin/mysql -S /run/mysqld/mysqld.sock -P 3307 mysql -p'MX4DdZze' -e 'select host,user,password from user;'
+---------------------------+--------------+-------------------------------------------+
| host | user | password |
+---------------------------+--------------+-------------------------------------------+
| localhost | root | *3D51EF7DDA58F435DE1F2512B9AFA1D4BEC97D6F |
| node-1.test.domain.local | root | *3D51EF7DDA58F435DE1F2512B9AFA1D4BEC97D6F |
| 127.0.0.1 | root | *3D51EF7DDA58F435DE1F2512B9AFA1D4BEC97D6F |
| ::1 | root | *3D51EF7DDA58F435DE1F2512B9AFA1D4BEC97D6F |
| % | wsrep_sst | *3D51EF7DDA58F435DE1F2512B9AFA1D4BEC97D6F |
| localhost | wsrep_sst | *3D51EF7DDA58F435DE1F2512B9AFA1D4BEC97D6F |
| 10.109.37.0/255.255.255.0 | root | *3D51EF7DDA58F435DE1F2512B9AFA1D4BEC97D6F |
| 10.109.39.6 | clustercheck | *0C8848611A5956D3789943A85F364CF2D428618F |
| % | keystone | *DC95241B069650FBFBC3C90AC76F63ABA8BBB2DF |
+---------------------------+--------------+-------------------------------------------+
root@node-1:~# fgrep 'mgmt/database' /etc/astute.yaml
        mgmt/database: 10.109.39.5
        mgmt/database: 10.109.39.4
        mgmt/database: 10.109.39.6
        mgmt/database: 10.109.39.3
        mgmt/database: 10.109.39.2
    mgmt/database: br-storage
root@node-1:~# ip netns exec haproxy ip -o -4 a
1: lo inet 127.0.0.1/8 scope host lo\ valid_lft forever preferred_lft forever
35: hapr-m inet 10.109.37.3/24 scope global hapr-m\ valid_lft forever preferred_lft forever
37: hapr-p inet 10.109.36.3/24 scope global hapr-p\ valid_lft forever preferred_lft forever
39: hapr-ns inet 240.0.0.2/30 scope global hapr-ns\ valid_lft forever preferred_lft forever
root@node-1:~# ip netns exec haproxy ip r g 10.109.39.6
10.109.39.6 via 240.0.0.1 dev hapr-ns src 240.0.0.2

As you can see 'root' user can log in to MySQL only from localhost or 'Management' network IP. Unfortunately changing of host '10.109.37.0/255.255.255.0' to '10.109.39.0/255.255.255.0' doesn't resolve the problem - HAProxy network namespace isn't directly connected to 'Storage' network, so it goes to MySQL through 240.0.0.2. So IMHO the simplest way to solve the issue is configure access for 'root'@'240.0.0.2' user in MySQL by default.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/206451

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/206451
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=0fdac52df103f55452e610e1fc6531e812122751
Submitter: Jenkins
Branch: master

commit 0fdac52df103f55452e610e1fc6531e812122751
Author: Oleksiy Molchanov <email address hidden>
Date: Tue Jul 28 12:58:39 2015 +0300

    Allow namespace host IPs to access Mysql

    Allow namespace host IPs '240.0.0.0/255.255.255.0'
    to access Mysql as root user

    Change-Id: I4ffcbcbd2cb31e66f542198c403eea69a14b810c
    Closes-Bug: 1478858

[Artem Panchenko (apanchenko-8)]

verified on iso #157

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "7.0"
  openstack_version: "2015.1.0-7.0"
  api: "1.0"
  build_number: "157"
  build_id: "2015-08-10_19-24-50"
  nailgun_sha: "9a6ac9b08733ded67bc65345a37787886ff4249d"
  python-fuelclient_sha: "ec5c02b3848fe1d15e5b50b323c3dda030f8e1ac"
  fuel-agent_sha: "57145b1d8804389304cd04322ba0fb3dc9d30327"
  fuel-nailgun-agent_sha: "e01693992d7a0304d926b922b43f3b747c35964c"
  astute_sha: "e1d3a435e5df5b40cbfb1a3acf80b4176d15a2dc"
  fuel-library_sha: "c4b2b9a1b464d7f538635cbfc9304a714ededc41"
  fuel-ostf_sha: "c7f745431aa3c147f2491c865e029e0ffea91c47"
  fuelmain_sha: "826387d1a05a40e4849433598442024bb1e3b7c7"