kernel 3.13.0-32 ipvs "IPv6 header not found" related to UDP socket sendto() EPERM errors

Bug #1349768 reported by Tero Marttila on 2014-07-29
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Medium
Chris J Arges
Trusty
Medium
Chris J Arges
Utopic
Medium
Chris J Arges

Bug Description

SRU Justificaton:

[Impact]
Users of ipvs may encounter dropped packets and message such as:
[70325.516724] IPv6 header not found

[Fix]
Commit eb90b0c734ad793d5f5bf230a9e9a4dcc48df8aa.

[Regression Potential]
This changes NFPROTO_IPV4 to NFPROTO_IPV6 as we'd expect this nf_hook_ops struct to be in the code.

--

I have an Ubuntu 14.04 host that I am using as both a keepalived/ipvs loadbalancer and dnsmasq server for pxebooting servers.

After updating linux-image 3.13.0-29.53 -> 3.13.0-32.57 I noticed that dnsmasq-tftp stopped working. pxeboot clients would hang on the "Loading ..../linux" TFTP transfer, with the transfer stalling roughly ~1000 blocks into the transfer:

10:30:51.011728 IP 10.1.1.2.43540 > 10.1.12.1.49165: UDP, length 1412
10:30:51.011924 IP 10.1.12.1.49165 > 10.1.1.2.43540: UDP, length 4
10:30:51.012012 IP 10.1.1.2.43540 > 10.1.12.1.49165: UDP, length 1412
10:30:51.012183 IP 10.1.12.1.49165 > 10.1.1.2.43540: UDP, length 4

stracing dnsmasq I noticed something very odd: sendto() on the socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) would suddenly start persistently returning EPERM in mid-transfer, even when dnsmasq continued to periodically retry:

select(18, [4 5 6 7 8 9 10 11 12 15 17], [], [], {0, 250000}) = 1 (in [17], left {0, 249834})
recvfrom(17, "\0\4\3\352", 4096, 0, NULL, NULL) = 4
lseek(16, 1410816, SEEK_SET) = 1410816
read(16, "\25\306\345f\2{\r\4)W\276\32\336q\252_\230q\213\341U\354\25\374k7\243\32\221X+\v"..., 1408) = 1408
sendto(17, "\0\3\3\353\25\306\345f\2{\r\4)W\276\32\336q\252_\230q\213\341U\354\25\374k7\243\32"..., 1412, 0, {sa_family=AF_INET, sin_port=htons(49165), sin_addr=inet_addr("10.1.11.3")}, 16) = 1412
select(18, [4 5 6 7 8 9 10 11 12 15 17], [], [], {0, 250000}) = 1 (in [17], left {0, 249839})
recvfrom(17, "\0\4\3\353", 4096, 0, NULL, NULL) = 4
lseek(16, 1412224, SEEK_SET) = 1412224
read(16, "*\360 <C\363l\320:\256~\307\236\26P\323\274%\260\362\341&\232\r\243\370\224\277\221\\\307\372"..., 1408) = 1408
sendto(17, "\0\3\3\354*\360 <C\363l\320:\256~\307\236\26P\323\274%\260\362\341&\232\r\243\370\224\277"..., 1412, 0, {sa_family=AF_INET, sin_port=htons(49165), sin_addr=inet_addr("10.1.11.3")}, 16) = -1 EPERM (Operation not permitted)
select(18, [4 5 6 7 8 9 10 11 12 15 17], [], [], {0, 250000}) = 0 (Timeout)
select(18, [4 5 6 7 8 9 10 11 12 15 17], [], [], {0, 250000}) = 0 (Timeout)
select(18, [4 5 6 7 8 9 10 11 12 15 17], [], [], {0, 250000}) = 0 (Timeout)
select(18, [4 5 6 7 8 9 10 11 12 15 17], [], [], {0, 250000}) = 0 (Timeout)
select(18, [4 5 6 7 8 9 10 11 12 15 17], [], [], {0, 250000}) = 0 (Timeout)
select(18, [4 5 6 7 8 9 10 11 12 15 17], [], [], {0, 250000}) = 0 (Timeout)
select(18, [4 5 6 7 8 9 10 11 12 15 17], [], [], {0, 250000}) = 0 (Timeout)
select(18, [4 5 6 7 8 9 10 11 12 15 17], [], [], {0, 250000}) = 0 (Timeout)
lseek(16, 1412224, SEEK_SET) = 1412224
read(16, "*\360 <C\363l\320:\256~\307\236\26P\323\274%\260\362\341&\232\r\243\370\224\277\221\\\307\372"..., 1408) = 1408
sendto(17, "\0\3\3\354*\360 <C\363l\320:\256~\307\236\26P\323\274%\260\362\341&\232\r\243\370\224\277"..., 1412, 0, {sa_family=AF_INET, sin_port=htons(49165), sin_addr=inet_addr("10.1.11.3")}, 16) = -1 EPERM (Operation not permitted)

This was with all iptables rules unloaded (so no OUTPUT -j DENY) and apparmor profiles torn down.

I also noticed the following dmesgs appearing at roughly similar times to the tftp transfers getting stuck (although not coinciding exactly with the stall):

[70325.516724] IPv6 header not found

The error pointed to ipvs (which I am using on the same host as an IPv4 NAT loadbalancer):
http://archive.linuxvirtualserver.org/html/lvs-devel/2012-08/msg00018.html
http://comments.gmane.org/gmane.comp.linux.lvs.devel/3614

I then tore down the ipvs rules (service keepalived stop) and unloaded the modules (rmmod ip_vs_rr ip_vs), and the issue resolved itself - the stalled dnsmasq-tftp transfer resumed!

This seems to be reproducible, i.e. modprobing ip_vs and starting keepalived will cause dnsmasq-tftp to stall again, and stopping/unloading will resume.

This seems to happen reproducibly on boot with -32 and -30. This does NOT seem to happen with 3.13.0-29 which I was using up until now.
---
AlsaDevices:
 total 0
 crw-rw---- 1 root audio 116, 1 Jul 29 13:43 seq
 crw-rw---- 1 root audio 116, 33 Jul 29 13:43 timer
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 2.14.1-0ubuntu3.2
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: Error: [Errno 2] No such file or directory
DistroRelease: Ubuntu 14.04
HibernationDevice: RESUME=/dev/mapper/catcp2-swap
InstallationDate: Installed on 2014-06-03 (56 days ago)
InstallationMedia: Ubuntu-Server 14.04 LTS "Trusty Tahr" - Release amd64 (20140416.2)
MachineType: Dell Inc. PowerEdge R410
Package: linux-image-3.13.0-32-generic 3.13.0-32.57
PackageArchitecture: amd64
PciMultimedia:

ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 LANG=C.UTF-8
 SHELL=/bin/bash
ProcFB:

ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.13.0-32-generic root=/dev/mapper/hostname-root ro console=ttyS1,115200n8 console=tty0 nomdmonddf nomdmonisw
ProcVersionSignature: Ubuntu 3.13.0-32.57-generic 3.13.11.4
RelatedPackageVersions:
 linux-restricted-modules-3.13.0-32-generic N/A
 linux-backports-modules-3.13.0-32-generic N/A
 linux-firmware 1.127.5
RfKill: Error: [Errno 2] No such file or directory
Tags: trusty
Uname: Linux 3.13.0-32-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:

_MarkForUpload: True
dmi.bios.date: 07/30/2013
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 1.12.0
dmi.board.name: 01V648
dmi.board.vendor: Dell Inc.
dmi.board.version: A03
dmi.chassis.type: 23
dmi.chassis.vendor: Dell Inc.
dmi.modalias: dmi:bvnDellInc.:bvr1.12.0:bd07/30/2013:svnDellInc.:pnPowerEdgeR410:pvr:rvnDellInc.:rn01V648:rvrA03:cvnDellInc.:ct23:cvr:
dmi.product.name: PowerEdge R410
dmi.sys.vendor: Dell Inc.

Tero Marttila (terom) wrote :

Apologies, misfiled initially on biosdevname, I meant to file on the linux-image package source.

no longer affects: biosdevname (Ubuntu)
description: updated

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1349768

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete

apport information

tags: added: apport-collected trusty
description: updated

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

apport information

Changed in linux (Ubuntu):
status: Incomplete → Confirmed
Tero Marttila (terom) wrote :

Reported apport-collect information against package linux-image-3.13.0-32-generic, reproducing the dnsmasq-tftp issue with a remote client pxebooting and stalling.

The CurrentDmesg contains the symptom message at 1214.740931

Tim Gardner (timg-tpi) on 2014-07-29
Changed in linux (Ubuntu Trusty):
assignee: nobody → Chris J Arges (arges)
status: New → In Progress
Changed in linux (Ubuntu Utopic):
status: Confirmed → Fix Released
Chris J Arges (arges) wrote :

@terom:
Can you test the following build:
http://people.canonical.com/~arges/lp1349768/

I've reverted commit: b05f2cbc367eae3ad8de17c65a832fd10e30f4b5 from ubuntu-trusty which was a backport of 57a7744e09867ebcfa0ccf1d6d529caa7728d552.

Thanks,

Changed in linux (Ubuntu Trusty):
importance: Undecided → Medium
Changed in linux (Ubuntu Utopic):
importance: Undecided → Medium
Tero Marttila (terom) wrote :

NACK

The same faulty behaviour persists with linux-image-3.13.0-33-generic=3.13.0-33.58~lp1349768v201407290941 (uname 3.13.0-33-generic #58~lp1349768v201407290941).

I do not notice any difference; identical symptoms. `rmmod ip_vs_rr ip_vs` continues to immediately resolve the fault.

Note that my configuration also includes openvswitch. The tftp stall continues to happen when flushing the ipvs ruleset, only unloading the ip_vs module resolves it. However, only re-loading the ip_vs_rr module does not trigger the stall, but loading the ipvs ruleset does. My IPVS ruleset consists of only four TCP "Masq" forwards.

Using ktrace to try and get a rough stack trace, I was able to observe the following sequences:

         dnsmasq-5012 [001] .... 2060.572946: ipv6_find_hdr <-ip_vs_out.part.23
         dnsmasq-5012 [001] .... 2060.572955: <stack trace>
 => ip_vs_local_reply6
 => nf_iterate
 => nf_hook_slow
 => __ip_local_out
 => ip_local_out
 => ip_send_skb
 => udp_send_skb
 => udp_sendmsg
 => inet_sendmsg
 => sock_sendmsg
 => SYSC_sendto
 => SyS_sendto
 => tracesys
         dnsmasq-5012 [001] .... 2060.572956: ipv6_find_hdr <-ip_vs_out_icmp_v6.isra.22
         dnsmasq-5012 [001] .... 2060.572963: <stack trace>
 => ip_vs_out.part.23
 => ip_vs_local_reply6
 => nf_iterate
 => nf_hook_slow
 => __ip_local_out
 => ip_local_out
 => ip_send_skb
 => udp_send_skb
 => udp_sendmsg
 => inet_sendmsg
 => sock_sendmsg
 => SYSC_sendto
 => SyS_sendto
 => tracesys

with the first sequence being repeated for every packet (?) and the second, slightly different, sequence corresponding per the timestamp to the dmesg:

[ 2060.572964] IPv6 header not found

This is with a my full iptables rulset loaded, but my OUTPUT DROP rules counters do not indicate any matches. The ftrace output seems to be identical with the iptables ruleset flushed.

Chris J Arges (arges) wrote :

I believe this is related to a CONFIG change:
UBUNTU: [Config] Enable CONFIG_IP_VS_IPV6=y

I'm building a 3.13 kernel with this CONFIG reverted I would like you to test and confirm.
(I'll post a link in a few hours)

In the meantime can you verify if the utopic 3.16 series kernel is also affected? (since it has the same CONFIG):
https://launchpad.net/ubuntu/+source/linux/3.16.0-6.11/+build/6217368

Thanks,

Changed in linux (Ubuntu Utopic):
status: Fix Released → New

This change was made by a bot.

Changed in linux (Ubuntu):
status: New → Confirmed
Chris J Arges (arges) wrote :

Ok here is the 3.13 kernel with CONFIG_IP_VS_IPV6=y reverted:
http://people.canonical.com/~arges/lp1349768v2/

Please let me know the results when you can.

In addition, if there is a way to explain a minimal reproducer of this issue, I could set this up here and run experiments as well.

Thanks,

Tero Marttila (terom) wrote :

ACK

I can confirm that this issue is no longer present with the Linux 3.13.0-33-generic #58~lp1349768v2v201407300928 kernel as installed from above. Rebooting into the given kernel results in dnsmasq-tftp transfers working fine, and no related dmesg errors occur.

Apologies for the delay. I was away, and had to reproduce this in a test environment first.

Do tell me if you would still like a minimal reproducer to track down the issue further, and I can work on getting something like that up. AFAICT it would involve dnsmasq with enable-tftp/tftp-root and a libvirt-qemu domain with <boot dev='network'> , requesting the ubuntu-installer amd64 linux kernel over TFTP, with the ip_vs modules loaded and a trivial ipvs ruleset applied on the server.

Tero Marttila (terom) wrote :

The utopic kernel appears to be affected as well. Booting the same Ubuntu 14.04 machine into the Linux 3.16.0-6-generic #11-Ubuntu kernel installed from

   https://launchpad.net/ubuntu/+source/linux/3.16.0-6.11/+build/6217368

gives identical symptoms, i.e. dnsmasq-tftp stalls, dmesg reports "IPv6 header not found", stracing dnsmasq shows sendto() giving EPERM, and unloading ip_vs allows the stalled tftp transfer to continue.

Chris J Arges (arges) wrote :

@terom
Yes, a minimal reproducer would help so I can debug on this end. I really appreciate your testing and analysis so far. : )
I did setup a quick netboot server using MAAS and it did seem to work; but I did not do the additional steps of setting up the proper ipvs ruleset to reproduce the issue. I also wonder if those rules can be set and something as simple as a ping could be used to illustrate the bug?

Here is a patch that looks like it could fix this issue; but unfortunately this was never placed upstream:
http://archive.linuxvirtualserver.org/cgi-bin/mesg.cgi?a=lvs-devel&i=20140219123150.GF20771%40telegraafnet.nl

I've built a test build with that patch applied for 3.13, let me know if this is an ACK or NAK in the meantime:
http://people.canonical.com/~arges/lp1349768v3/

Thanks

Tero Marttila (terom) wrote :

NAK

Linux 3.13.0-35-generic #62~lp1349768v3v201408191448 still exhibits the same symptoms (stalled dnsmasq-tftp transfer and "IPv6 header not found" messages). Unloading the ip_vs module clears the issue.

I'll attempt to work out a minimal reproducer.

Tero Marttila (terom) wrote :

Attached is a small script to setup dnsmasq-tftp and ipvs to reproduce the dnsmasq-tftp stall. The strace at the end should start showing sendto() = EPERM errors after running the curl tftp://.../ command given as an example from a remote host.

The exact contents of the ipvs rules don't seem to matter.

The dnsmasq-tftp stall issue appears to happen with all the ubuntu-installer/amd64/linux versions that I've tried, with the exact block within the TFTP stream where it stalls changing betwen different versions but remaining consistent between runs.

With the current netboot.tar.gz it appers to happen at block 600:

d4fc2ba26cad96f4360cb36f04c26b2f50dceee9413c206431148b87e4958909 /srv/tftp/ubuntu-installer/amd64/linux

With the original ubuntu-installer netboot.tar.gz version (from sometime early June) I've been using it happens at block 311:

bdebc6c7bbd873bf5ad2480d90c2523da49cca8da5a0f4cb40adb7d1aeafb821 /srv/tftp/lp1349768/linux

Curiously, however, testing this setup on the 3.13.0-34-generic kernel the dmesg error ("IPv6 header not found") does NOT show up... may be some other facters at play as well there?

Chris J Arges (arges) wrote :

@terom

Alright! I've been able to reproduce this issue, I'll start debugging this today.

Regarding the 3.13.0-34 differences I can look at the patches in that kernel to see if there was some changes to ipvs or around where that message shows up.

Chris J Arges (arges) on 2014-08-20
tags: added: bug-exists-upstream
Changed in linux (Ubuntu Utopic):
assignee: nobody → Chris J Arges (arges)
status: Confirmed → In Progress
Chris J Arges (arges) wrote :

@terom

Can you please test the following build:
http://people.canonical.com/~arges/lp1349768v4/

This has the patch identified by upstream; it would be good to have positive testing on this.
This has VS_IPV6 enabled with the fix:
http://archive.linuxvirtualserver.org/html/lvs-devel/2014-08/msg00027.html

Thanks,

Tero Marttila (terom) wrote :

NAK

Unfortunately no, the 3.13.0-35-generic #62~lp1349768v4v201408220857 from above continues to exhibit the same issues with my actual test setup. Same thing with the ./lp1349768.sh testcase.

The 0041-ipvs-fix-ipv4-issues-in-ip_vs_out.patch in the lp1349768v4 build is a different patch, though?

The resulting ftrace that correlates with dnsmasq sendto() getting stuck is still the same:

         dnsmasq-1447 [000] .... 1272.430975: ipv6_find_hdr <-ip_vs_out_icmp_v6.isra.21
         dnsmasq-1447 [000] .... 1272.430983: <stack trace>
 => ip_vs_out.part.22
 => ip_vs_local_reply6
 => nf_iterate
 => nf_hook_slow
 => __ip_local_out
 => ip_local_out
 => ip_send_skb
 => udp_send_skb
 => udp_sendmsg
 => inet_sendmsg
 => sock_sendmsg
 => SYSC_sendto
 => SyS_sendto
 => tracesys

 Is there something I can use to confirm that the booted kernel image actually has the correct nf_hook_ops.pf set? The fix seems pretty obvious, so I was expecting it to work...

Chris J Arges (arges) wrote :

@terom,
Well, I posted the wrong kernel! I'll post the correct one soon after it builds. Thanks

Chris J Arges (arges) wrote :

This should be the right kernel:
http://people.canonical.com/~arges/lp1349768v5/

I verified it myself and it works, once you verify, I'll go through the SRU process and get this into Trusty.
Thanks,

Tero Marttila (terom) wrote :

ACK

3.13.0-35-generic #62~lp1349768v5v201408250842 appears to resolve the issue completely on my actual test setup as well. No TFTP stalls, dnsmasq EPERM errors or dmesg errors, and ftrace doesn't show any calls to ipv6_find_hdr.

Chris J Arges (arges) wrote :

Ok patch is upstream in v3.17-rc5:
eb90b0c734ad793d5f5bf230a9e9a4dcc48df8aa

Chris J Arges (arges) on 2014-09-24
description: updated
Tim Gardner (timg-tpi) on 2014-09-24
Changed in linux (Ubuntu Utopic):
status: In Progress → Fix Committed
Changed in linux (Ubuntu Trusty):
status: In Progress → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package linux - 3.16.0-18.25

---------------
linux (3.16.0-18.25) utopic; urgency=low

  [ Tim Gardner ]

  * Release Tracking Bug
    - LP: #1373682
 -- Tim Gardner <email address hidden> Wed, 24 Sep 2014 19:23:23 -0600

Changed in linux (Ubuntu Utopic):
status: Fix Committed → Fix Released
Brad Figg (brad-figg) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-trusty' to 'verification-done-trusty'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-trusty
Tero Marttila (terom-u) wrote :

Yes, I can confirm that the trusty-proposed 3.13.0-38 kernel resolves this issue. The dnsmasq-tftp transfer succeeds in my test environment without any of the symptoms that were present with the trusty-updates/security kernels between -30 and -37, nor do I notice any regressions in my (IPv4) ipvs ruleset.

$ uname -a
Linux catcp2-test 3.13.0-38-generic #65-Ubuntu SMP Thu Oct 9 11:36:50 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

$ apt-cache policy linux-image-generic
linux-image-generic:
  Installed: 3.13.0.38.45
  Candidate: 3.13.0.38.45
  Version table:
 *** 3.13.0.38.45 0
         50 http://fi.archive.ubuntu.com/ubuntu/ trusty-proposed/main amd64 Packages
        100 /var/lib/dpkg/status
  ...

tags: added: verification-done-trusty
removed: verification-needed-trusty
Chris J Arges (arges) wrote :

@terom-u
Thanks for verifying this fix!
--chris

Launchpad Janitor (janitor) wrote :
Download full text (10.4 KiB)

This bug was fixed in the package linux - 3.13.0-39.66

---------------
linux (3.13.0-39.66) trusty; urgency=low

  [ Luis Henriques ]

  * Release Tracking Bug
    - LP: #1386629

  [ Upstream Kernel Changes ]

  * KVM: x86: Check non-canonical addresses upon WRMSR
    - LP: #1384539
    - CVE-2014-3610
  * KVM: x86: Prevent host from panicking on shared MSR writes.
    - LP: #1384539
    - CVE-2014-3610
  * KVM: x86: Improve thread safety in pit
    - LP: #1384540
    - CVE-2014-3611
  * KVM: x86: Fix wrong masking on relative jump/call
    - LP: #1384545
    - CVE-2014-3647
  * KVM: x86: Warn if guest virtual address space is not 48-bits
    - LP: #1384545
    - CVE-2014-3647
  * KVM: x86: Emulator fixes for eip canonical checks on near branches
    - LP: #1384545
    - CVE-2014-3647
  * KVM: x86: emulating descriptor load misses long-mode case
    - LP: #1384545
    - CVE-2014-3647
  * KVM: x86: Handle errors when RIP is set during far jumps
    - LP: #1384545
    - CVE-2014-3647
  * kvm: vmx: handle invvpid vm exit gracefully
    - LP: #1384544
    - CVE-2014-3646
  * Input: synaptics - gate forcepad support by DMI check
    - LP: #1381815

linux (3.13.0-38.65) trusty; urgency=low

  [ Luis Henriques ]

  * Release Tracking Bug
    - LP: #1379244

  [ Andy Whitcroft ]

  * Revert "SAUCE: scsi: hyper-v storsvc switch up to SPC-3"
    - LP: #1354397
  * [Config] linux-image-extra is additive to linux-image
    - LP: #1375310
  * [Config] linux-image-extra postrm is not needed on purge
    - LP: #1375310

  [ Upstream Kernel Changes ]

  * Revert "KVM: x86: Increase the number of fixed MTRR regs to 10"
    - LP: #1377564
  * Revert "USB: option,zte_ev: move most ZTE CDMA devices to zte_ev"
    - LP: #1377564
  * aufs: bugfix, stop calling security_mmap_file() again
    - LP: #1371316
  * ipvs: fix ipv6 hook registration for local replies
    - LP: #1349768
  * Drivers: add blist flags
    - LP: #1354397
  * sd: fix a bug in deriving the FLUSH_TIMEOUT from the basic I/O timeout
    - LP: #1354397
  * drm/i915/bdw: Add 42ms delay for IPS disable
    - LP: #1374389
  * drm/i915: add null render states for gen6, gen7 and gen8
    - LP: #1374389
  * drm/i915/bdw: 3D_CHICKEN3 has write mask bits
    - LP: #1374389
  * drm/i915/bdw: Disable idle DOP clock gating
    - LP: #1374389
  * drm/i915: call lpt_init_clock_gating on BDW too
    - LP: #1374389
  * drm/i915: shuffle panel code
    - LP: #1374389
  * drm/i915: extract backlight minimum brightness from VBT
    - LP: #1374389
  * drm/i915: respect the VBT minimum backlight brightness
    - LP: #1374389
  * drm/i915/bdw: Apply workarounds in render ring init function
    - LP: #1374389
  * drm/i915/bdw: Cleanup pre prod workarounds
    - LP: #1374389
  * drm/i915: Replace hardcoded cacheline size with macro
    - LP: #1374389
  * drm/i915: Refactor Broadwell PIPE_CONTROL emission into a helper.
    - LP: #1374389
  * drm/i915: Add the WaCsStallBeforeStateCacheInvalidate:bdw workaround.
    - LP: #1374389
  * drm/i915/bdw: Remove BDW preproduction W/As until C stepping.
    - LP: #1374389
  * mptfusion: enable no_write_same for vmware scsi disks
    - LP: #1371591
  * iommu/amd: Fix cleanup_domai...

Changed in linux (Ubuntu Trusty):
status: Fix Committed → Fix Released
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers