QoS egress minimum bandwidth with Linuxbridge doesn't work

Bug #1662582 reported by Hirofumi Ichihara
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
Fix Released
Critical
Rodolfo Alonso

Bug Description

I tested QoS egress minimum bandwidth with Linuxbridge but it doesn't work in my env.

My test scenario:
1) prepares 2 compute nodes(nodeA, nodeB) which are connected 100Mbps.
2) creates network1(with minimum bandwidth rule 70Mbps) and network2(non rule)
3) creates VM1(on nodeA) and VM2(on nodeB) with network1, and VM3(on nodeA) and VM4(on nodeB) with network2
4) runs iperf from VM1 to VM2 and from VM3 to VM4

In (4), I expected VM1-VM2's result is over 70Mbps and VM3-VM4's result is below 30Mbps because of minimum bandwidth rule. However, they were almost the same value(40~45Mbps).

The following is a example of commands that I executed in openstack:
neutron net-create network1
neutron subnet-create network1 10.0.1.0/24 --name subnet1
neutron qos-policy-create bw-min
neutron qos-minimum-bandwidth-rule-create --min-kbps 70000 --direction egress bw-min
neutron net-update network1 --qos-policy bw-min
neutron net-create network2
neutron subnet-create network2 10.0.1.0/24 --name subnet2
nova boot VM1~4

The following is a example of commands that I executed in VM1,3:
$ iperf -c $IP_ADDRESS_FOR_VM2_OR_VM4 -u -b 100M -t 10 -i 1

The following is a example of commands that I executed in VM2,4:
$ iperf -s -u

When I executed iperf in one side only, the result was 84.2Mbps.

I saw the code a little. It seems that minimum bandwidth rule is set in the ifb of each ports. Traffic from VM1 and VM3 is filtered by each ifb(I guess that the rule allows the traffic to be 100Mbps) and then it overflows in the NIC of nodeA.

I believe that the issue is just issue of my env. Could you QoS folks point out my mistake?

tags: added: qos
Changed in neutron:
assignee: nobody → Rodolfo Alonso (rodolfo-alonso-hernandez)
Revision history for this message
Rodolfo Alonso (rodolfo-alonso-hernandez) wrote :

Hello Ichihara.

I made some tests in a fresh environment and I discovered two things:
1) I didn't setup correctly the default qos queue in the qdisc. This is also affecting max-bw. The solution is to set the default queue to 1.

2) For the min-bw: the qos policies are applied for each tap port, but there is no common shaper for all the flows. That's the way should be done.

When I made my tests I did all of them in the same environment with manual policies applied. That's why I thought it was working. I'M VERY SORRY FOR THAT.

For now, the solution should be to remove the min-bw support for Linux Bridge until a new solution is proposed.

Revision history for this message
Hirofumi Ichihara (ichihara-hirofumi) wrote :

Hi Rodolfo,

Don't mind. QoS logic is very complicated and we core reviewers missed it too.
However, we should not release the feature at Ocata. I proposed revert patch[1].
The patch may depend on other patches so it's difficult to remove it.
I'd like Ajo and Kevin to judge what we should.

[1]:https://review.openstack.org/#/c/431506/

Changed in neutron:
importance: Undecided → Critical
status: New → Confirmed
Changed in neutron:
milestone: none → ocata-rc2
Revision history for this message
Miguel Angel Ajo (mangelajo) wrote :

After talking to Rodolfo Alonso, we believe that reverting is a good way to go,
because, when the ifb structure is fixed, we would also need to clean up the old
ifbs on the system.

By introducing the fixed version in Pike we save us some trouble,
and we don't miss lead any customer about the features.

Revision history for this message
Miguel Angel Ajo (mangelajo) wrote :

If we are not in time for revert, we can fix the max-bw as ralonsoh proposes, keep the bug open, fix it in pike and backport.

but we will need to add some check to cleanup old ifbs.

Revision history for this message
Ihar Hrachyshka (ihar-hrachyshka) wrote :

Can't we just patch out the advertisement of the support for the rule for LB, and leave the rest of the code in place? Then backport a fix later.

tags: added: ocata-rc-potential
Revision history for this message
Kevin Benton (kevinbenton) wrote :
Revision history for this message
Armando Migliaccio (armando-migliaccio) wrote :
Changed in neutron:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/ocata)

Reviewed: https://review.openstack.org/431784
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=0eb7bf97cc18aea4dc8ea221ca44e1828f806fb7
Submitter: Jenkins
Branch: stable/ocata

commit 0eb7bf97cc18aea4dc8ea221ca44e1828f806fb7
Author: Hirofumi Ichihara <email address hidden>
Date: Thu Feb 9 12:59:04 2017 +0000

    Revert "Linux Bridge: driver support for QoS egress minimum bandwidth"

    This reverts commit 84b3ae3ae9111e241352bdccad238c9fd37b83df.

    The logic was incorrect[1]. We cannot achieve QoS egress minimum bandwidth of Linuxbridge by the patch. I also think that the issue is not solved by small patch and we must consider deeply.

    Closes-bug: 1662582

    (cherry picked from commit e3063496cfc6fa39ac6d2b76da0d876c8dfdebd2)

    Change-Id: Id4703b5c63876f16e31b6805cd147b5840a4a591

tags: added: in-stable-ocata
Changed in neutron:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 10.0.0.0rc2

This issue was fixed in the openstack/neutron 10.0.0.0rc2 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.