[FFE] DPDK: fix vfio-pci poll mode driver probing on ppc64el

[bugproxy]

[FFE]
IMHO not fully being an FFE, but wanted to double check as we are really late in the cycle. The doc says "f you have doubts about if something qualifies, check with a member of ubuntu-release (or subscribe ubuntu-release to the bug)".
Background of this is that we enabled DPDK (and openvswitch-dpdk) for ppc64el in the zesty cycle. IBM committed to work on upstream to make this better supported. Part of that is work is the patch in this bug (and its sibling bug 1670686). That will make more of the HW available on power working properly with DPDK.
For FFE ack speaks:
- essentially it is "fixing" vfio-poll to work on ppc64el
- only minimally touching other arches
- without it the new ppc64el enablement is a bit of a lackluster
Against FFE:
- On some Point of views one could argue that this is a "feature" and it is too late (which is why I made this an FFE to get an ack)

----

Request to enable ppc64le support for DPDK poll mode drivers probing PCI over vfio-pci kernel module.

DPDK Upstream patch for this feature is available as:
http://dpdk.org/dev/patchwork/patch/21482/

We would like to bring this patch in Ubuntu due to its importance for DPDK poll mode drivers in ppc64le.

[Christian Ehrhardt  (paelzer)]

We should name that like "fix vfio-pci poll mode driver probing on ppc64el" because that is what it is - a fix to get that working. Even this in some sense is not so much a feature IMHO as it is there but faulty.

Note: backport from dpdk 17.05

[Christian Ehrhardt  (paelzer)]

Quoting from https://wiki.ubuntu.com/FreezeExceptionProcess#FeatureFreeze_for_bugfix-only_updates

"Up through RC, if a developer believes an upload of a new upstream release that just has bug fixes in it is warranted, they may upload it. The developer should explicitly document that this is a bugfix-only upload in the changelog or sync request.

If you have doubts about if something qualifies, check with a member of ubuntu-release (or subscribe ubuntu-release to the bug) and if one person from ubuntu-release agrees it's a bug fix update, you're good for upload."

I think here this applies, also changes are pretty much pcc64el only masked behind the RTE_VFIO_SPAPR.
Yet since we are post Feature Freeze I'd like to have an ack of the Release Team (subscribing them later)

[Christian Ehrhardt  (paelzer)]

Note: sibling to bug 1670686

[Christian Ehrhardt  (paelzer)]

Discussed on IRC:
- I will prepare a build with that on bileto
- gshankarmk will test on ppc64el
- I will test on x86
- Upstream has to fully accept the patches (one is only on multi-ack atm)
- the Release Team has to ack FFE on this (and the sibling)

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-03-07 09:59 EDT-------
(In reply to comment #9)
> Discussed on IRC:
> - I will prepare a build with that on bileto
> - gshankarmk will test on ppc64el
> - I will test on x86
> - Upstream has to fully accept the patches (one is only on multi-ack atm)
> - the Release Team has to ack FFE on this (and the sibling)

Thanks for your support on this Christian!

[Christian Ehrhardt  (paelzer)]

Suggested upload started to build in https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/2546

Please test as we discussed

[Christian Ehrhardt  (paelzer)]

@Release-Team
Could you please take a look and consider ack'ing this FFE.
It is essentially a fix as I outlined above, but we are late so I wanted to be sure.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-03-08 07:26 EDT-------
> Suggested upload started to build in
> https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/2546
>
> Please test as we discussed

I have tested i40e pmd using this patch through testpmd binary

~$ uname -a
Linux ip9-114-219-126gate 4.10.0-11-generic #13-Ubuntu SMP Wed Mar 1 21:24:32 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux

~$ sudo testpmd -l 0,8,16,40,48
EAL: Detected 10 lcore(s)
EAL: No free hugepages reported in hugepages-16777216kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: PCI device 0004:01:00.0 on NUMA socket 1
EAL: probe driver: 8086:1583 net_i40e
EAL: using IOMMU type 7 (sPAPR)
PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.05 eetrack 800028a6
EAL: PCI device 0004:01:00.1 on NUMA socket 1
EAL: probe driver: 8086:1583 net_i40e
PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.05 eetrack 800028a6
USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=179456, size=2176, socket=0
Configuring Port 0 (socket 0)
Port 0: 68:05:CA:2F:70:E0
Configuring Port 1 (socket 0)
Port 1: 68:05:CA:2F:70:E1
Checking link statuses...
Port 0 Link Up - speed 40000 Mbps - full-duplex
Port 1 Link Up - speed 40000 Mbps - full-duplex
Done
No commandline core given, start packet forwarding
io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support disabled, MP over anonymous pages disabled
Logical Core 8 (socket 0) forwards packets on 2 streams:
RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=02:00:00:00:00:01
RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00

io packet forwarding - CRC stripping disabled - packets/burst=32
nb forwarding cores=1 - nb forwarding ports=2
RX queues=1 - RX desc=128 - RX free threshold=32
RX threshold registers: pthresh=8 hthresh=8 wthresh=0
TX queues=1 - TX desc=512 - TX free threshold=32
TX threshold registers: pthresh=32 hthresh=0 wthresh=0
TX RS bit threshold=32 - TXQ flags=0xf01
Press enter to exit

Telling cores to stop...
Waiting for lcores to finish...

---------------------- Forward statistics for port 0 ----------------------
RX-packets: 12352398 RX-dropped: 0 RX-total: 12352398
TX-packets: 349033 TX-dropped: 0 TX-total: 349033
----------------------------------------------------------------------------

---------------------- Forward statistics for port 1 ----------------------
RX-packets: 349033 RX-dropped: 0 RX-total: 349033
TX-packets: 12352398 TX-dropped: 0 TX-total: 12352398
----------------------------------------------------------------------------

+++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++
RX-packets: 12701431 RX-dropped: 0 RX-total: 12701431
TX-packets: 12701431 TX-dropped: 0 TX-total: 12701431
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Done.

Shutting down port 0...
Stopping ports...
Done
Closing ports...
Done

Shutting down port 1...
Stopping ports...
Done
Closing ports...
Done

Bye...

$ dpkg -l | grep dpdk
ii dpdk 16.11.1-0ubuntu3 ...

[Christian Ehrhardt  (paelzer)]

Also x86 based tests completed on these:
- dep8 anyway via the bileto build
- testpmd tests
- openvswitch-dpdk performance tests
- openvswitch-dpdk tuned performance tests
- endurance start-stop dpdk using guests
- endurance add remove dpdk ports

Also as mentioned http://dpdk.org/browse/dpdk/commit/?id=0fe9830b53452a6747cae9ff1a6bfc737b839a9d
is upstream already in the main repo.

That said I think all is ready for pushing to proposed now, except an ack for the little remaining FFE character on this - pinging release Team in IRC about it to speed that up.

[Andy Whitcroft (apw)]

These two fixes (bug 1670686 and bug 1670689) represent bringing a feature
to ppc64el. The feature patches look small, self-contained and specific
to the ppc64el architecture. The new code is upstream and backported to
the zesty version. It brings feature parity to ppc64el.

Regression potential is very limited as the changes are architecture specific
utilising a h/w feature only available on ppc64el. Existing architectures
should be unmodified and ppc64el (as stated above) is a new architecture here
so there is no regression potential their either.

The package with these patches has passed dep8 testing (marginal value)
and manual testing both for the new ppc64el packages and on amd64 (an
existing architecture).

This is a new feature which will be a tech-preview in zesty, so we have
time to iterate should there be any issues. The ubuntu-server team and
IBM are committed to bug fixing this support.

Acking (on behalf of the release-team) for upload to zesty-proposed.

[Christian Ehrhardt  (paelzer)]

Thanks, since all was ready and tested I just had to press publish on the Bileto ticket now, setting to "fix committed" and waiting for an automated "fix released" when it is hopefully through z-p just nice.

[Launchpad Janitor (janitor)]

This bug was fixed in the package dpdk - 16.11.1-0ubuntu3

---------------
dpdk (16.11.1-0ubuntu3) zesty; urgency=medium

  * debian/control: enable librte-pmd-i40e1 for ppc64el

 -- Christian Ehrhardt <email address hidden> Tue, 07 Mar 2017 17:00:08 +0100

[Christian Ehrhardt  (paelzer)]

Back then we rejected the ppc64el enablement "in archive" in the yakktey release, referring that we wanted to see a better commitment to get DPDK on ppc64el properly working and supported.
Your work became just that, good, responding in time, reachable for discussions and participating with us and upstream.
That said, I wanted to close thanking gshankarmk for making that happen!

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2017-03-10 19:02 EDT-------
(In reply to comment #17)
> Back then we rejected the ppc64el enablement "in archive" in the yakktey
> release, referring that we wanted to see a better commitment to get DPDK on
> ppc64el properly working and supported.
> Your work became just that, good, responding in time, reachable for
> discussions and participating with us and upstream.
> That said, I wanted to close thanking gshankarmk for making that happen!

Thanks to you too Christian for enabling this to happen!

[Christian Ehrhardt  (paelzer)]

As discussed on IRC with release Team and then IBM I added a statement that calls it "tech preview" to the release notes. That on one hand makes people aware that it is now there, but OTOH also allows us to inject bigger SRU fixes if they might be needed as this is kind of a first time support.