test_create_port_in_allowed_allocation_pools: No more IP addresses available

Bug #1514457 reported by Guillaume Chenuet
20
This bug affects 4 people
Affects Status Importance Assigned to Milestone
tempest
Fix Released
Undecided
Guillaume Chenuet

Bug Description

When running Tempest test 'tempest.api.network.test_ports.PortsTestJSON.test_create_port_in_allowed_allocation_pools' on a 3 controllers infra, we have this error: 'No more IP addresses available on network UUID'.

As we have 3 controllers, each one pick an IP and the pools have no more IP available.

In the code, the function created a new subnet with only 3 IPs available for the allocation pools. (ex: 192.168.1.4 - 192.168.1.6).

-----
pythonlogging:'': {{{
2015-11-09 11:49:18,795 22399 INFO [tempest_lib.common.rest_client] Request (PortsTestJSON:test_create_port_in_allowed_allocation_pools): 200 POST http://10.225.195.69:5000/v2.0/tokens
2015-11-09 11:49:18,796 22399 DEBUG [tempest_lib.common.rest_client] Request - Headers: {}
        Body: None
    Response - Headers: {'status': '200', 'content-length': '4199', 'vary': 'X-Auth-Token', 'connection': 'close', 'date': 'Mon, 09 Nov 2015 14:46:54 GMT', 'content-type': 'application/json', 'x-openstack-request-id': 'req-22d341c1-6beb-4fe7-8ad1-b5eff6d52823'}
        Body: None
2015-11-09 11:49:18,948 22399 INFO [tempest_lib.common.rest_client] Request (PortsTestJSON:test_create_port_in_allowed_allocation_pools): 201 POST http://10.225.195.69:9696/v2.0/networks 0.152s
2015-11-09 11:49:18,948 22399 DEBUG [tempest_lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: {"network": {"name": "test-network--1178376881"}}
    Response - Headers: {'status': '201', 'content-length': '256', 'connection': 'close', 'date': 'Mon, 09 Nov 2015 14:46:54 GMT', 'content-type': 'application/json; charset=UTF-8', 'x-openstack-request-id': 'req-27f2cf9b-727e-4719-bc9a-d03e3330013c'}
        Body: {"network": {"status": "ACTIVE", "subnets": [], "name": "test-network--1178376881", "router:external": false, "tenant_id": "17b606473f8943659d2645ee5f7ff95a", "admin_state_up": true, "mtu": 0, "shared": false, "id": "12e0345d-f3c2-431f-b2ab-650d9eeb63b8"}}
2015-11-09 11:49:19,051 22399 INFO [tempest_lib.common.rest_client] Request (PortsTestJSON:test_create_port_in_allowed_allocation_pools): 200 POST http://10.225.195.69:5000/v2.0/tokens
2015-11-09 11:49:19,051 22399 DEBUG [tempest_lib.common.rest_client] Request - Headers: {}
        Body: None
    Response - Headers: {'status': '200', 'content-length': '4199', 'vary': 'X-Auth-Token', 'connection': 'close', 'date': 'Mon, 09 Nov 2015 14:46:54 GMT', 'content-type': 'application/json', 'x-openstack-request-id': 'req-ac3ea446-cd81-4a09-adb0-24b0b2224b3b'}
        Body: None
2015-11-09 11:49:19,306 22399 INFO [tempest_lib.common.rest_client] Request (PortsTestJSON:test_create_port_in_allowed_allocation_pools): 201 POST http://10.225.195.69:9696/v2.0/subnets 0.254s
2015-11-09 11:49:19,307 22399 DEBUG [tempest_lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: {"subnet": {"ip_version": 4, "network_id": "12e0345d-f3c2-431f-b2ab-650d9eeb63b8", "allocation_pools": [{"start": "10.100.0.4", "end": "10.100.0.6"}], "gateway_ip": "10.100.0.1", "cidr": "10.100.0.0/28"}}
    Response - Headers: {'status': '201', 'content-length': '446', 'connection': 'close', 'date': 'Mon, 09 Nov 2015 14:46:54 GMT', 'content-type': 'application/json; charset=UTF-8', 'x-openstack-request-id': 'req-4be58f0b-affd-410b-aa5a-59038fc1a887'}
        Body: {"subnet": {"name": "", "enable_dhcp": true, "network_id": "12e0345d-f3c2-431f-b2ab-650d9eeb63b8", "tenant_id": "17b606473f8943659d2645ee5f7ff95a", "dns_nameservers": [], "gateway_ip": "10.100.0.1", "ipv6_ra_mode": null, "allocation_pools": [{"start": "10.100.0.4", "end": "10.100.0.6"}], "host_routes": [], "ip_version": 4, "ipv6_address_mode": null, "cidr": "10.100.0.0/28", "id": "4ef9d8bf-4c40-4894-a9e1-157b11066299", "subnetpool_id": null}}
2015-11-09 11:49:19,430 22399 INFO [tempest_lib.common.rest_client] Request (PortsTestJSON:test_create_port_in_allowed_allocation_pools): 200 POST http://10.225.195.69:5000/v2.0/tokens
2015-11-09 11:49:19,431 22399 DEBUG [tempest_lib.common.rest_client] Request - Headers: {}
        Body: None
    Response - Headers: {'status': '200', 'content-length': '4199', 'vary': 'X-Auth-Token', 'connection': 'close', 'date': 'Mon, 09 Nov 2015 14:46:54 GMT', 'content-type': 'application/json', 'x-openstack-request-id': 'req-5da73564-055d-462a-a4e4-a5b7f530a027'}
        Body: None
2015-11-09 11:49:19,622 22399 INFO [tempest_lib.common.rest_client] Request (PortsTestJSON:test_create_port_in_allowed_allocation_pools): 409 POST http://10.225.195.69:9696/v2.0/ports 0.191s
2015-11-09 11:49:19,622 22399 DEBUG [tempest_lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: {"port": {"network_id": "12e0345d-f3c2-431f-b2ab-650d9eeb63b8"}}
    Response - Headers: {'status': '409', 'content-length': '164', 'connection': 'close', 'date': 'Mon, 09 Nov 2015 14:46:54 GMT', 'content-type': 'application/json; charset=UTF-8', 'x-openstack-request-id': 'req-c425640c-9816-47b5-bbfb-b0c2e14d5c52'}
        Body: {"NeutronError": {"message": "No more IP addresses available on network 12e0345d-f3c2-431f-b2ab-650d9eeb63b8.", "type": "IpAddressGenerationFailure", "detail": ""}}
2015-11-09 11:49:19,734 22399 INFO [tempest_lib.common.rest_client] Request (PortsTestJSON:_run_cleanups): 200 POST http://10.225.195.69:5000/v2.0/tokens
2015-11-09 11:49:19,735 22399 DEBUG [tempest_lib.common.rest_client] Request - Headers: {}
        Body: None
    Response - Headers: {'status': '200', 'content-length': '4199', 'vary': 'X-Auth-Token', 'connection': 'close', 'date': 'Mon, 09 Nov 2015 14:46:55 GMT', 'content-type': 'application/json', 'x-openstack-request-id': 'req-560d0b8b-0b2e-4256-8acd-7547433f95f7'}
        Body: None
2015-11-09 11:49:20,312 22399 INFO [tempest_lib.common.rest_client] Request (PortsTestJSON:_run_cleanups): 204 DELETE http://10.225.195.69:9696/v2.0/subnets/4ef9d8bf-4c40-4894-a9e1-157b11066299 0.576s
2015-11-09 11:49:20,312 22399 DEBUG [tempest_lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'date': 'Mon, 09 Nov 2015 14:46:55 GMT', 'status': '204', 'connection': 'close', 'content-length': '0', 'x-openstack-request-id': 'req-0d07dcca-a156-4de4-b6d1-6497d3f20732'}
        Body:
}}}

Traceback (most recent call last):
  File "/home/stack/openstack-deployment/tempest/tempest/api/network/test_ports.py", line 110, in test_create_port_in_allowed_allocation_pools
    body = self.client.create_port(network_id=net_id)
  File "/home/stack/openstack-deployment/tempest/tempest/services/network/json/network_client.py", line 139, in _create
    resp, body = self.post(uri, post_data)
  File "/usr/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 252, in post
    return self.request('POST', url, extra_headers, headers, body)
  File "/usr/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 629, in request
    resp, resp_body)
  File "/usr/lib/python2.7/site-packages/tempest_lib/common/rest_client.py", line 685, in _error_checker
    raise exceptions.Conflict(resp_body)
Conflict: An object with that identifier already exists
Details: {u'message': u'No more IP addresses available on network 12e0345d-f3c2-431f-b2ab-650d9eeb63b8.', u'type': u'IpAddressGenerationFailure', u'detail': u''}

description: updated
Changed in tempest:
assignee: nobody → Guillaume Chenuet (gchenuet)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tempest (master)

Fix proposed to branch: master
Review: https://review.openstack.org/243649

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/243652

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/243660

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on tempest (master)

Change abandoned by Guillaume Chenuet (<email address hidden>) on branch: master
Review: https://review.openstack.org/243649
Reason: Duplicate review

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Change abandoned by Guillaume Chenuet (<email address hidden>) on branch: master
Review: https://review.openstack.org/243652
Reason: Duplicate review

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Change abandoned by Guillaume Chenuet (<email address hidden>) on branch: master
Review: https://review.openstack.org/243660
Reason: Duplicate review

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tempest (master)

Fix proposed to branch: master
Review: https://review.openstack.org/249182

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on tempest (master)

Change abandoned by Guillaume Chenuet (<email address hidden>) on branch: master
Review: https://review.openstack.org/249182

Revision history for this message
Viktor Tikkanen (viktor-tikkanen) wrote :

It seems that the tool has missed following proposed fix:

https://review.openstack.org/243159/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tempest (master)

Reviewed: https://review.openstack.org/243159
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=bdf1d8d11fe57074214d639715eb229a83b52325
Submitter: Jenkins
Branch: master

commit bdf1d8d11fe57074214d639715eb229a83b52325
Author: Guillaume Chenuet <email address hidden>
Date: Mon Nov 9 16:10:48 2015 +0100

    Fixing test_create_port_in_allowed_allocation_pool when 3 (or more) controllers

    When running Tempest test test_create_port_in_allowed_allocation_pools
    on a 3 controllers infra, we have this error: No more IP addresses available
    on network UUID.

    In the code, the function created a new subnet with only 3 IPs available for
    the allocation pools.

    One is used for the network, another for neutron s port and the last one for
    the broadcast.

    A better way could be to used and concate default parameters
    tenant_network_subnet and tenant_network_mask_bits and check if the subnet
    is large enough.

    Changed way to calculate allocation_pool:

    * Used and concate tenant_network_subnet & tenant_network_mask_bits
    * Check if the mask bits is not too small for the test
    * If it is OK, create net, subnet and try to allocate a port

    Change-Id: Id523c54037518ea199cd88d553b4cdfef59a186b
    Closes-Bug: 1514457

Changed in tempest:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.