[doc-tools] Cannot fully generate neutron config reference

[Atsushi SAKAI]

Cannot fully generate neutron config plugin

At the time of writing, there are 3 problem exists

1)a few plugins are supported.
   a)Need to update requirements/neutron-master.txt
     AND to update plugin list on extra_repos/neutron-master

   b)Also extra_repos are assumed git://git.openstack.org/openstack/xxxxxxx
     it shoukd suppprt git://git.openstack.org/stackforge/xxxxxxxxx

2)Cannot import networking_ibm.sdnve.common.config
    With autohelp.py with -v (verbose) option,
    Failed to import: networking_ibm.sdnve.common.config

  As a workaround, I treated with manual handling.

3)Some networking-xxxx object name convension is differnt from
  autohelp-wrapper expectation.

  for example (correct one)
    networking-cisco's second layer class name is
    networking_cisco.

  The problem example (5 plugins)
    networking-midonet midonet
    networking-hyperv hyperv
    networking-bigswitch bsnstacklib
    networking-edge-vpn networking-edge-vpn
    networking-zvm neutron

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/232358

[Atsushi SAKAI (sakaia)]

Problem still remain

item 2 still exists

item 4 (same as item2)
import failed
  networking-zvm cannot import

item 5
docbook generation failed
  networkinb-infoblox
     infoblox-dc:xxxxxx xxxx=data center id
     causes generation problem

  This item little bit different since "infoxxxx" is static in general.
  But infoblox uses dynamic one

  see
  DATA_CENTER_SECTION = 'infoblox-dc:%s' % CONF_IPAM.cloud_data_center_id

  https://github.com/openstack/networking-infoblox/blob/5efc6a7b67cc229a4179a298d858ed996de73ae3/networking_infoblox/neutron/common/config.py#L34

[Atsushi SAKAI (sakaia)]

note for #2

for item 2, previous one exists so keep previous one
for item 4, previous one does not exist. so it does not exist on document.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/220118
Committed: https://git.openstack.org/cgit/openstack/openstack-manuals/commit/?id=71bf779f1c8c7b4f34e1d0dfe4215a538a5b00f1
Submitter: Jenkins
Branch: master

commit 71bf779f1c8c7b4f34e1d0dfe4215a538a5b00f1
Author: Atsushi SAKAI <email address hidden>
Date: Tue Oct 13 16:56:59 2015 +0900

    Update Config reference for neutron

      Edit section_network-plugins.xml
           section_network-plugins-ml2.xml
      Stay removed mlnx (mellanox)

      Unchanged because of trouble
           networking-ibm (SDN-VE)
           octavia

      Cannot include because of trouble
           networking-infoblox
           networking-zvm

      Consider neutron plugin addition (#1503967)

    Change-Id: I41272af4ec46c7e7c1c3ce7d6e03c6a40fb0cb25
    Partial-Bug: #1407581
    Related-Bug: #1503967

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/232358
Committed: https://git.openstack.org/cgit/openstack/openstack-doc-tools/commit/?id=c77c706b293c4f4c99a68bc489791ee09b6e6030
Submitter: Jenkins
Branch: master

commit c77c706b293c4f4c99a68bc489791ee09b6e6030
Author: Atsushi SAKAI <email address hidden>
Date: Wed Oct 7 18:20:38 2015 +0900

    Add neutron plugin modules for Config Reference autogen

      Item 1 in #1503967)
        Add extra_repos/neutron-master for many plugins
        Add extra_repos/neutron-master-stackforge for stackforge plugins
          also edit autohelp-wrapper to handle stackforge url

      Item 3 in #1503967)
        Add object name re-mapping for
         networking-midonet
         networking-hyperv
         networking-bigswitch
         networking-edge-vpn
         networking-zvm
       on autohelp-wrapper

    Change-Id: Ib247fed088446c79b7d9e6a9e1adbbc931108a3b
    Partial-Bug: #1503967

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/236703

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/236703
Committed: https://git.openstack.org/cgit/openstack/openstack-doc-tools/commit/?id=3dbacea2fbcfa2bdb6a88e0cf4ab72b704308efe
Submitter: Jenkins
Branch: master

commit 3dbacea2fbcfa2bdb6a88e0cf4ab72b704308efe
Author: Atsushi SAKAI <email address hidden>
Date: Sun Oct 18 13:23:34 2015 +0000

    Update config reference package list for neutron

     This is just for update package list changes.
     Not solved config reference autogeneration problem.

     Remove stackforge package handling in autohelp-wrapper
     Remove modules
       networking-ibm (SDNVE) (It seems no maintained)
       networking-bigswitch (It seems no maintained)
       networking-portforwarding (It seems no maintained)

     Changes from stackforge to openstack
       networking-bagpipe-l2, networking-brocade,
       networking-edge-vpn, networking-hyperv,
       networking-mlnx, networking-nec, networking-ovs-dpdk,
       networking-zvm, networking-6wind

     Add
       octavia

    Change-Id: I5ba3b26e110f55b2cc76f37b903a1b07dd6a1202
    Related-Bug: #1503967

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/237463

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/237463
Committed: https://git.openstack.org/cgit/openstack/openstack-doc-tools/commit/?id=e30f841a2b8502c37e297d18a309e9a9804f018e
Submitter: Jenkins
Branch: master

commit e30f841a2b8502c37e297d18a309e9a9804f018e
Author: Atsushi SAKAI <email address hidden>
Date: Tue Oct 20 15:52:35 2015 +0900

    Sorting in config-reference autogeneration list for neutron

      sorting
      in requirements/neutron-master.txt

      removing stackforge/networking-bigswitch
      in requirements/neutron-master.txt
         autohelp-wrapper (name space conversion)

    Change-Id: I29d0465f03b21f31f76c2a81abc4db4e1ccc3e52
    Related-Bug: #1503967

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/237848

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/237851

[Atsushi SAKAI (sakaia)]

For Cisco package (APICAPI) solving, it needs to solve following issue.
oslo.config problem

https://github.com/noironetworks/apicapi/issues/9

[Atsushi SAKAI (sakaia)]

Remaining issues, (as a view of "No module named XXXXX")

For plumgridlib
  Need to find plumgrid lib

For brocade
 brocade_neutron_lbaas modules are not found
  even in following directories
  https://github.com/brocade-vadx/adx-device-driver/tree/master/brocade_neutron_lbaas
    adx_device_driver_v2.py is not found

For kemptechnologies
  need to contact for fixing oslo.config problem

For zvm
  Workaround is already supplied, but fix is not supplied.

Other remaining issues are

Failed to import: networking_infoblox.neutron.cmd.eventlet (No module named cmd.eventlet)
Failed to import: networking_infoblox.neutron.cmd.eventlet.infoblox_ipam_agent (No module named cmd.eventlet.infoblox_ipam_agent)
Failed to import: midonet.neutron.client.cluster (No module named _protobuf)
Failed to import: midonet.neutron.rpc.topology_client (No module named _protobuf)
Failed to import: networking_ovs_dpdk.agent.ovs_dpdk_firewall (No module named agent.ovs_dpdk_firewall)

Requested fix is (just reporting or sending patch)
  networking-odl
  networking-cisco (apicapi)

[Atsushi SAKAI (sakaia)]

#13 remain files are __init__.py location problem.
networking-infoblox is already posted a patch
networking-ovs-dpdk is NOT posted because of error (stackforge mover cannot receive patch at this moment?).

Failed to import: networking_infoblox.neutron.cmd.eventlet (No module named cmd.eventlet)
Failed to import: networking_infoblox.neutron.cmd.eventlet.infoblox_ipam_agent (No module named cmd.eventlet.infoblox_ipam_agent)
Failed to import: midonet.neutron.client.cluster (No module named _protobuf)
Failed to import: midonet.neutron.rpc.topology_client (No module named _protobuf)
Failed to import: networking_ovs_dpdk.agent.ovs_dpdk_firewall (No module named agent.ovs_dpdk_firewall)

So it seems this issue near to close.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/237851
Committed: https://git.openstack.org/cgit/openstack/openstack-doc-tools/commit/?id=6df87147054ed0a6587d532883867efb9b4dce59
Submitter: Jenkins
Branch: master

commit 6df87147054ed0a6587d532883867efb9b4dce59
Author: Atsushi SAKAI <email address hidden>
Date: Wed Oct 21 11:34:21 2015 +0900

    Add config reference autogeneration support for neutron-plugins

    Adds entry to requirements/neutron-master.txt
        apicapi, pika for cisco plugin
        nuage for neutron itself
        github/pypowervm for powervm plugin
        pyroute2 for octavia plugin

    With this fixes, almost problem for config-ref of neutron
    is solved.
    but another module problem is still requested to change
     apicapi oslo.config issue
     kemptechnologies oslo.config issue
     networking-ovs-dpdk __init__.py issue

    Change-Id: Id193307f338f008858edf401fadf9ea9b260ac3c
    Closes-Bug: #1503967
    Related-Bug: #1507479

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/238774

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/238774
Committed: https://git.openstack.org/cgit/openstack/openstack-doc-tools/commit/?id=5b391da4bc9b572734a9178eb992c1a49fe5c86d
Submitter: Jenkins
Branch: master

commit 5b391da4bc9b572734a9178eb992c1a49fe5c86d
Author: Atsushi SAKAI <email address hidden>
Date: Fri Oct 23 14:12:21 2015 +0900

    Update networking-cisco plugin requirements

    apicapi 1.0.4 does not support oslo.config new description.
      (i.e changes from oslo.config to oslo_config)
    but updated apicapi is already supported.
    switch from pip to github.

    Change-Id: I6f9802b4a8b8737b2fa3697150a973c046b9ea30
    Related-Bug: #1503967

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/239313

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/239313
Committed: https://git.openstack.org/cgit/openstack/openstack-doc-tools/commit/?id=da3e57d974d976fef06d1a6d7dc27c5d0c9577df
Submitter: Jenkins
Branch: master

commit da3e57d974d976fef06d1a6d7dc27c5d0c9577df
Author: Atsushi SAKAI <email address hidden>
Date: Mon Oct 26 11:35:03 2015 +0900

    Update kemptech lbaas package requirements for config reference

      Updates kemptech lbaas requirements,
      With the help of kemptech package updating by Shane McGough

    Change-Id: I9cceef081bb1a3f5f51540cd1b03d604974bdddc
    Related-Bug: #1503967

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/240922

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/240922
Committed: https://git.openstack.org/cgit/openstack/openstack-doc-tools/commit/?id=87b92ccf97f7953dbdb8cee1ddb5d8a15bfc1401
Submitter: Jenkins
Branch: master

commit 87b92ccf97f7953dbdb8cee1ddb5d8a15bfc1401
Author: Atsushi SAKAI <email address hidden>
Date: Mon Nov 2 16:13:32 2015 +0000

    Add kuryr and dragonflow to neutron config ref

    Updating package list based on following page
    http://governance.openstack.org/reference/projects/neutron.html

    Change-Id: If50a37dbaefc08141b7265cc7b7fc266c66f108f
    Related-Bug: #1503967

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/237848
Committed: https://git.openstack.org/cgit/openstack/openstack-doc-tools/commit/?id=a1a7ee5842b946dee7c0182a7b49e571c684cc5c
Submitter: Jenkins
Branch: master

commit a1a7ee5842b946dee7c0182a7b49e571c684cc5c
Author: Atsushi SAKAI <email address hidden>
Date: Wed Oct 21 11:06:38 2015 +0900

    config reference for neutron (networking-zvm)

      As a workaround, networking-zvm can generate config reference
      by directory copying to neutron tree in source directory.

      This patch will be reverted after name space fixation
      in networking-zvm.

      currently name space header uses "neutron",
      This causes conflict with neutron itself.

    Change-Id: Ia86eabf8c982431dbf6af727373349d3c2add1a9
    Related-Bug: #1503967