Duplicate MAC Addresses generated during Create Server stress tests

Bug #1050924 reported by Rohit Karajgi
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Unmesh Gurjar

Bug Description

We are running a Tempest stress script that calls the Create Server API in a loop to generate load on Nova.

During this, we encountered an error quite a few times where the network manager generates a MAC address that is already in the DB and fails to spawn the instance.
The issue is nova.utils.generate_mac_address() method generates a duplicate MAC address.

Error Log: http://paste.openstack.org/show/20933/

Revision history for this message
Thierry Carrez (ttx) wrote :

MAC addresses are generated with only 20 pseudo-random bits, so collisions are definitely /possible/. I'm surprised that would occur that often though. Maybe we can increase the number of random bits...

Changed in nova:
importance: Undecided → Medium
status: New → Confirmed
Revision history for this message
MotoKen (motokentsai) wrote :
Revision history for this message
Gary Kotton (garyk) wrote :

Quantum has the following solution:
1. The user is able to define how many bits he/she want to be used for the generation
2. In addition to this Quantum checks against the database if the MAC generated address has already been used (the user is able to define how many time generation can take place)
The following configuration variables are used:

# Base MAC address. The first 3 octets will remain unchanged. If the
# 4h octet is not 00, it will also used. The others will be
# randomly generated.
# 3 octet
# base_mac = fa:16:3e:00:00:00
# 4 octet
# base_mac = fa:16:3e:4f:00:00

# Maximum amount of retries to generate a unique MAC address
# mac_generation_retries = 16

Thanks
Gary

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/15183

Changed in nova:
assignee: nobody → Unmesh Gurjar (unmesh-gurjar)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/15183
Committed: http://github.com/openstack/nova/commit/f7433c85b9f04873404bcbe9dbb4aa228b835f8f
Submitter: Jenkins
Branch: master

commit f7433c85b9f04873404bcbe9dbb4aa228b835f8f
Author: Unmesh Gurjar <email address hidden>
Date: Wed Oct 31 22:31:46 2012 -0700

    Increased MAC address range to reduce conflicts

    Increased the 4th octet range from 7f to ff, making it consistent with
    Quantum. This also reduces the probability of a MAC address conflict.

    Fixes LP: #1050924

    Change-Id: I801a852110c0a35e5a6a31f359df75475a21fd4a

Changed in nova:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in nova:
milestone: none → grizzly-1
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in nova:
milestone: grizzly-1 → 2013.1
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.