Kernel 4.13-rc1 with custom patch for troubleshooting

[Patrik Kullman]

I kindly got offered to have a kernel built for me with a custom patch.

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Created attachment 251631
dmesg

I've installed Fedora 25 on a Dell XPS 13 9365 (2 in 1) laptop. Duel booted with Windows, secure boot is enabled, SATA mode is ACPI, and it boots fine. Everything works - sound, trackpad, wifi etc, without issue.

However suspending the machine does not work; the screen goes blank and it appears that the machine is entering sleep mode, but the power button remains lit, and it never seems to completely finish suspending. The behavior is the same with STI and STR, via these commands:

    echo > freeze /sys/power/state
    echo > mem /sys/power/state

Booting the kernel into runlevel 3 makes no difference - the behavior is the same. Disabling async module power down also did not change the behavior; using this command:

    echo 0 > /sys/power/pm_async

The kernel was booted with the following additional options: initcall_debug no_console_suspend ignore_loglevel 3 dyndbg=\\\"file suspend.c +p\\\".

No change in behavior was seen. Attached are the outputs of various

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Created attachment 251641
lspci

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Created attachment 251651
lsmod

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Created attachment 251661
boot-config

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Created attachment 251671
lspci

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Exact same behavior is seen under kernel 4.8.6-300.fc25.x86_64.

I should add: I can press the power button for a few seconds, and the system "resumes" (it doesn't appear to ever finish suspending). But closing/opening the lid, pressing keys or moving the trackpad does not resume. Only pressing the power button triggers the "resume".

[yu.c.chen (yu.c.chen-linux-kernel-bugs)]

Could you please check if it still exist on latest upstream kernel, we mainly track upstream version rather than distribute version.
If it is still there, you can use /sys/power/pm_test to figure out at which phase it failed. thanks.

[rui.zhang (rui.zhang-linux-kernel-bugs)]

(In reply to davidnmfarrell from comment #0)
> However suspending the machine does not work; the screen goes blank and it
> appears that the machine is entering sleep mode, but the power button
> remains lit,

blink or lit?

(In reply to davidnmfarrell from comment #5)
> Exact same behavior is seen under kernel 4.8.6-300.fc25.x86_64.
>
> I should add: I can press the power button for a few seconds, and the system
> "resumes" (it doesn't appear to ever finish suspending).

please attach the dmesg after the system "resumes"
we can check kernel log to see if it is a real suspend.

BTW, you said press the power button for a few seconds? what if you press the button and release it immediately?

> But closing/opening
> the lid, pressing keys or moving the trackpad does not resume. Only pressing
> the power button triggers the "resume".

For STI only or for both STR and STI?

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

(In reply to Zhang Rui from comment #7)

> blink or lit?

lit, constant light (no blinking)

> BTW, you said press the power button for a few seconds? what if you press
> the button and release it immediately?

Nothing happens

> For STI only or for both STR and STI?

For both STI and STR

I also see the same issue under newer kernels:

4.9.3-200.fc25.x86_64
4.10.0-0.rc3.fc26.x86_64

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Created attachment 251761
dmesg after suspend resume

[rui.zhang (rui.zhang-linux-kernel-bugs)]

please attach the output of "cat /proc/acpi/wakeup" as long as the dmesg output after long power button press.

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

(In reply to Zhang Rui from comment #7)

> please attach the dmesg after the system "resumes"
> we can check kernel log to see if it is a real suspend.

Sure thing, I've uploaded the file "dmesg after suspend resume"

[rui.zhang (rui.zhang-linux-kernel-bugs)]

(In reply to davidnmfarrell from comment #9)
> Created attachment 251761 [details]
> dmesg after suspend resume

I don't think this is the dmesg output after suspend/resume as I can see nothing indicating a suspend is started.
please make a double check.
You can also check the dmesg before echo mem/freeze > /sys/power/state, and then attach the dmesg after the system is back and tell me if there is something new.

[rui.zhang (rui.zhang-linux-kernel-bugs)]

(In reply to davidnmfarrell from comment #9)
> Created attachment 251761 [details]
> dmesg after suspend resume

For me, this looks like a dmesg output after a fresh boot.

(In reply to davidnmfarrell from comment #5)
> Exact same behavior is seen under kernel 4.8.6-300.fc25.x86_64.
>
> I should add: I can press the power button for a few seconds, and the system
> "resumes" (it doesn't appear to ever finish suspending).

when you say "resumes", does the system restore back to the same state before "suspend", or it just boots?

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Created attachment 251771
proc-acpi-wakeup

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Created attachment 251781
dmesg-before-suspend

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Created attachment 251791
dmesg-after-suspend-resume

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

(In reply to Zhang Rui from comment #12)
> I don't think this is the dmesg output after suspend/resume as I can see
> nothing indicating a suspend is started.
> please make a double check.

No problem - I uploaded before & after dmesg files

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

(In reply to Zhang Rui from comment #13)

> when you say "resumes", does the system restore back to the same state
> before "suspend", or it just boots?

It restores to the same state as before suspending.

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Created attachment 251811
mod-params

All loaded modules and their parameters

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

I tried removing modules and then suspending, but it made no difference:

modprobe -r iwlmvm cfg80211 mac80211 iwlwifi
modprobe -r wacom
modprobe -r acpi_als intel_lpss_acpi int3400_thermal acpi_pad acpi_thermal_rel

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

I tried:

analyze_suspend.py -rtcwake 30 -f -m mem

As detailed here:
https://01.org/blogs/rzhang/2015/best-practice-debug-linux-suspend/hibernate-issues

Unusually it looked to me like the suspend worked - the screen went blank, the power button turned off, etc. I'll upload the output files.

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Created attachment 251821
analyse-suspend-dmesg

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Ok, I had a chance to compare the behavior of my machine (9365) against another XPS 13 (9360) that works with STI/STR. tldr; suspend seems to work, it looks like resume works, but the screen does not turn back on.

STI
9360: Screen turns off, power button light remains on. Pressing the power button immediately resumes to original state.

9365: Screen turns off, power button light remains on. Pressing power button seems to have no effect. Pressing and holding power button for ~8 seconds the screen comes back on, original state is resumed (run level 3 it stays on, run level 5 screen immediately goes off again).

STR
9360: Screen turns off, power button light goes off. Pressing the power button immediately resumes: power button light comes on, screen comes back. Original state is resumed.

9365: Screen turns off, power button light goes off. Pressing power button, the power button light comes back on, but the screen remains off. Pressing and holding power button for ~8 seconds the screen comes back on (run level 3 it stays on, run level 5 it immediately goes off again).

I'm attaching dmesg logs for pre and post STR.

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Created attachment 251851
dmesg-pre-str

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Created attachment 251861
dmesg-post-str

[rui.zhang (rui.zhang-linux-kernel-bugs)]

at runtime, please attach the output of "grep . /sys/firmware/acpi/interrupts/*" and "cat /proc/interrupts"
1. before do nothing
2. after press and release the power button
3. after press and hold the power button for 8 seconds.

[rui.zhang (rui.zhang-linux-kernel-bugs)]

To, this seems like a power button interrupt issue instead of an ACPI problem.

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Created attachment 251921
acpi-interrupts-pre-str

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

Created attachment 251931
acpi-interrupts-post-str

[davidnmfarrell (davidnmfarrell-linux-kernel-bugs)]

I uploaded the pre/post STR output. I wasn't able to run grep after pressing and releasing the power button as the screen and keyboard were disabled.

[bpshacklett (bpshacklett-linux-kernel-bugs)]

The testing branch still has numerous problems for me:

The laptop does come back on with an extended press to the power button, but sometimes the touchpad doesn't work.

Reopening the laptop or hitting keys on the keyboard doesn't wake up the laptop.

Sometimes the first press of the power button doesn't work and to wake it back up requires holding the button down, releasing and holding it down again. Like Patrik, I had to turn off Suspend on power button press, otherwise the laptop immediately suspends after waking up.

[superm1 (superm1-linux-kernel-bugs)]

@Brennan,
To be clear - are you using s2idle or s3?

[lenb (lenb-linux-kernel-bugs)]

We need to blacklist S3 on the Dell 9365, due to last of working BIOS support.

[bpshacklett (bpshacklett-linux-kernel-bugs)]

It seems that closing the laptop puts it into s3. I tested with s2idle with echo freeze > /sys/power/state, and there are a different set of issues:

- The power light does not turn off
- Keyboard, trackpad still don't wake the laptop back up
- Power button instantly turns the screen back on, but the laptop is totally frozen.

[superm1 (superm1-linux-kernel-bugs)]

@Brennan,

Please adjust the default the kernel uses by kernel command line parameter mem_sleep_default. This will let the correct action occur on the XPS 9365.

I agree with Len, either S3 should be blacklisted on 9365 or default policy should be quirked to s2idle.

Power light not turning off is an understood problem, and updates should be coming soon regarding that.

That last issue on freezing is news to me.

[matt (matt-linux-kernel-bugs)]

Adding a comment that with 4.12, mem_sleep_default=s2idle on the boot command line, and the latest linux-firmware package (1.167_all) the behavior of my XPS 13 9365 2 in 1 is close to "as expected" - suspend works normally, through GUI menu, command line, or closing the lid. Power button LED does eventually turn off, but I'm not sure how long it takes - more than one hour, less than six. Does not fully wake up without holding the power button for several (>5) seconds; opening the lid, pressing keys or trackpad turns on the keyboard backlight and sometimes the screen backlight but no interactive activity until the power button is pressed. However, once the power button is pressed for that several seconds, everything functions as expected.

[patrik.kullman (patrik.kullman-linux-kernel-bugs)]

Same experience as Matt, also excited about 4.13 since the last(?) needed fixes (acpi-pm-test branch) for a smooth experience seems to be merged in the 4.13-rc1 tree: https://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm.git/tag/?h=pm-4.13-rc1

[patrik.kullman (patrik.kullman-linux-kernel-bugs)]

Using 4.13-rc1 from http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.13-rc1/, suspend and resume works beautifully!

Thanks for the great work!

[patrik.kullman (patrik.kullman-linux-kernel-bugs)]

Actually it doesn't seem to suspend properly, I get a lot of these while suspended:

[48773.433130] Suspended for 1.736 seconds
[48773.487290] PM: noirq resume of devices complete after 54.007 msecs
[48773.584996] PM: noirq suspend of devices complete after 92.106 msecs
[48773.640021] PM: noirq resume of devices complete after 55.019 msecs
[48773.693010] PM: noirq suspend of devices complete after 52.909 msecs
[48774.772925] Suspended for 0.739 seconds
[48774.827173] PM: noirq resume of devices complete after 54.107 msecs
[48774.924789] PM: noirq suspend of devices complete after 92.105 msecs
[48774.979256] PM: noirq resume of devices complete after 54.460 msecs
[48775.040788] PM: noirq suspend of devices complete after 61.453 msecs
[48776.118507] Suspended for 1.732 seconds
[48776.173052] PM: noirq resume of devices complete after 54.401 msecs
[48776.274384] PM: noirq suspend of devices complete after 96.105 msecs
[48776.328516] PM: noirq resume of devices complete after 54.124 msecs
[48776.386397] PM: noirq suspend of devices complete after 57.801 msecs
[48777.458479] Suspended for 0.731 seconds
[48777.512360] PM: noirq resume of devices complete after 53.748 msecs
[48777.578332] PM: noirq suspend of devices complete after 60.094 msecs
[48777.639708] PM: noirq resume of devices complete after 61.369 msecs
[48777.698355] PM: noirq suspend of devices complete after 58.569 msecs
[48778.803968] Suspended for 0.760 seconds
[48778.858508] PM: noirq resume of devices complete after 54.374 msecs
[48778.955839] PM: noirq suspend of devices complete after 92.111 msecs
[48779.010944] PM: noirq resume of devices complete after 55.097 msecs
[48779.067844] PM: noirq suspend of devices complete after 56.823 msecs
[48780.143970] Suspended for 1.736 seconds
[48780.198045] PM: noirq resume of devices complete after 53.932 msecs
[48780.299838] PM: noirq suspend of devices complete after 96.105 msecs
[48780.354429] PM: noirq resume of devices complete after 54.583 msecs
[48780.415834] PM: noirq suspend of devices complete after 61.318 msecs
[48781.489487] Suspended for 0.728 seconds
[48781.543593] PM: noirq resume of devices complete after 53.953 msecs
[48781.641374] PM: noirq suspend of devices complete after 92.110 msecs
[48781.702691] PM: noirq resume of devices complete after 61.310 msecs
[48781.761389] PM: noirq suspend of devices complete after 58.618 msecs
[48782.829505] Suspended for 0.727 seconds
[48782.883630] PM: noirq resume of devices complete after 53.983 msecs
[48782.985371] PM: noirq suspend of devices complete after 96.111 msecs
[48783.039782] PM: noirq resume of devices complete after 54.403 msecs
[48783.089373] PM: noirq suspend of devices complete after 49.510 msecs
[48784.175236] Suspended for 1.740 seconds
[48784.230309] PM: noirq resume of devices complete after 54.926 msecs
[48784.323101] PM: noirq suspend of devices complete after 88.100 msecs
[48784.377473] PM: noirq resume of devices complete after 54.365 msecs
[48784.435113] PM: noirq suspend of devices complete after 57.558 msecs
[48785.515050] Suspended for 0.740 seconds
[48785.569140] PM: noirq resume of devices complete after 53.939 msecs
[48785.666922] PM: noirq suspend of devi...

[rjw (rjw-linux-kernel-bugs)]

(In reply to Patrik Kullman from comment #96)
> Actually it doesn't seem to suspend properly, I get a lot of these while
> suspended:

Well, that's how the platform works, unfortunately.

The EC needs to be enabled to wake up the system for the power button wakeup to work, but if the EC is enabled to wake up the system, it will wake it up every once a while, so either-or.

I would argue that this is an EC firmware bug on the XPS13 9365.

> [48773.433130] Suspended for 1.736 seconds
> [48773.487290] PM: noirq resume of devices complete after 54.007 msecs
> [48773.584996] PM: noirq suspend of devices complete after 92.106 msecs
> [48773.640021] PM: noirq resume of devices complete after 55.019 msecs
> [48773.693010] PM: noirq suspend of devices complete after 52.909 msecs
> [48774.772925] Suspended for 0.739 seconds
> [48774.827173] PM: noirq resume of devices complete after 54.107 msecs
> [48774.924789] PM: noirq suspend of devices complete after 92.105 msecs
> [48774.979256] PM: noirq resume of devices complete after 54.460 msecs
> [48775.040788] PM: noirq suspend of devices complete after 61.453 msecs
> [48776.118507] Suspended for 1.732 seconds
> [48776.173052] PM: noirq resume of devices complete after 54.401 msecs
> [48776.274384] PM: noirq suspend of devices complete after 96.105 msecs
> [48776.328516] PM: noirq resume of devices complete after 54.124 msecs
> [48776.386397] PM: noirq suspend of devices complete after 57.801 msecs
> [48777.458479] Suspended for 0.731 seconds
> [48777.512360] PM: noirq resume of devices complete after 53.748 msecs
> [48777.578332] PM: noirq suspend of devices complete after 60.094 msecs
> [48777.639708] PM: noirq resume of devices complete after 61.369 msecs
> [48777.698355] PM: noirq suspend of devices complete after 58.569 msecs
> [48778.803968] Suspended for 0.760 seconds
> [48778.858508] PM: noirq resume of devices complete after 54.374 msecs
> [48778.955839] PM: noirq suspend of devices complete after 92.111 msecs
> [48779.010944] PM: noirq resume of devices complete after 55.097 msecs
> [48779.067844] PM: noirq suspend of devices complete after 56.823 msecs
> [48780.143970] Suspended for 1.736 seconds
> [48780.198045] PM: noirq resume of devices complete after 53.932 msecs
> [48780.299838] PM: noirq suspend of devices complete after 96.105 msecs
> [48780.354429] PM: noirq resume of devices complete after 54.583 msecs
> [48780.415834] PM: noirq suspend of devices complete after 61.318 msecs
> [48781.489487] Suspended for 0.728 seconds
> [48781.543593] PM: noirq resume of devices complete after 53.953 msecs
> [48781.641374] PM: noirq suspend of devices complete after 92.110 msecs
> [48781.702691] PM: noirq resume of devices complete after 61.310 msecs
> [48781.761389] PM: noirq suspend of devices complete after 58.618 msecs
> [48782.829505] Suspended for 0.727 seconds
> [48782.883630] PM: noirq resume of devices complete after 53.983 msecs
> [48782.985371] PM: noirq suspend of devices complete after 96.111 msecs
> [48783.039782] PM: noirq resume of devices complete after 54.403 msecs
> [48783.089373] PM: noirq suspend of devices complete after 49.510 msecs
> [48784.175236] Suspended for 1.740 seconds
>...

[patrik.kullman (patrik.kullman-linux-kernel-bugs)]

(In reply to Rafael J. Wysocki from comment #97)
> (In reply to Patrik Kullman from comment #96)
> > Actually it doesn't seem to suspend properly, I get a lot of these while
> > suspended:
>
> Well, that's how the platform works, unfortunately.
>
> The EC needs to be enabled to wake up the system for the power button wakeup
> to work, but if the EC is enabled to wake up the system, it will wake it up
> every once a while, so either-or.

"every once a while" being every 0.7 - 1.7 seconds? :)
And I assume that it's no difference between the power button or the lid opening? (Looking for workarounds..)

> I would argue that this is an EC firmware bug on the XPS13 9365.

That it happens too frequently or at all?

> > And on resume I get a, most likely completely unrelated and purely
> cosmetic:
> >
> > [49569.968486] intel-vbtn INT33D6:00: unknown event index 0xcd
>
> I'm quite confident that this is not the event that woke you up, but can you
> paste more lines from dmesg around this for more context?

Here's a complete suspend/resume:

[ 432.314171] wlp60s0: deauthenticating from 08:60:6e:cb:73:18 by local choice (Reason: 3=DEAUTH_LEAVING)
[ 432.332499] wlp60s0: failed to remove key (2, ff:ff:ff:ff:ff:ff) from hardware (-22)
[ 432.344754] IPv6: ADDRCONF(NETDEV_UP): wlp60s0: link is not ready
[ 432.367450] ACPI Error: Thread 334907008 cannot release Mutex [PATM] acquired by thread 196679552 (20170531/exmutex-416)
[ 432.367461] ACPI Error: Method parse/execution failed \_SB.PCI0.LPCB.ECDV._Q66, AE_AML_NOT_OWNER (20170531/psparse-550)
[ 433.755730] PM: Syncing filesystems ... done.
[ 433.759972] PM: Preparing system for sleep (freeze)
[ 433.761479] Freezing user space processes ... (elapsed 0.002 seconds) done.
[ 433.764445] OOM killer disabled.
[ 433.764446] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
[ 433.766170] PM: Suspending system (freeze)
[ 433.766172] Suspending console(s) (use no_console_suspend to debug)
[ 434.196204] psmouse serio1: Failed to disable mouse on isa0060/serio1
[ 435.796225] PM: suspend of devices complete after 1815.500 msecs
[ 435.816250] PM: late suspend of devices complete after 20.016 msecs
[ 435.864129] PM: noirq suspend of devices complete after 45.221 msecs
[ 435.864133] PM: suspend-to-idle
[ 436.223170] Suspended for 0.049 seconds
[ 436.276728] PM: noirq resume of devices complete after 53.389 msecs
[ 436.375016] PM: noirq suspend of devices complete after 92.109 msecs
[ 436.431696] PM: noirq resume of devices complete after 56.675 msecs
[ 436.491007] PM: noirq suspend of devices complete after 59.227 msecs
[ 437.568679] Suspended for 0.732 seconds
[ 437.622933] PM: noirq resume of devices complete after 54.114 msecs
[ 437.720561] PM: noirq suspend of devices complete after 92.110 msecs
[ 437.774979] PM: noirq resume of devices complete after 54.411 msecs
[ 437.832567] PM: noirq suspend of devices complete after 57.513 msecs
[ 438.908663] Suspended for 0.735 seconds
[ 438.962854] PM: noirq resume of devices complete after 54.043 msecs
[ 439.060527] PM: noirq suspend of devices complete after 92.103 msecs
[ 439.114780] PM: noirq resume of devices complete a...

[rjw (rjw-linux-kernel-bugs)]

(In reply to Patrik Kullman from comment #98)
> (In reply to Rafael J. Wysocki from comment #97)
> > (In reply to Patrik Kullman from comment #96)
> > > Actually it doesn't seem to suspend properly, I get a lot of these while
> > > suspended:
> >
> > Well, that's how the platform works, unfortunately.
> >
> > The EC needs to be enabled to wake up the system for the power button
> wakeup
> > to work, but if the EC is enabled to wake up the system, it will wake it up
> > every once a while, so either-or.
>
> "every once a while" being every 0.7 - 1.7 seconds? :)

Yes, in this particular case ...

> And I assume that it's no difference between the power button or the lid
> opening? (Looking for workarounds..)

I'm not sure about the lid to be honest, let me try to dig into that somewhat.

> > I would argue that this is an EC firmware bug on the XPS13 9365.
>
> That it happens too frequently or at all?

It shouldn't wake you up at all in the state we have requested from it, but if it did that every minute, say, it probably wouldn't really matter.

> > > And on resume I get a, most likely completely unrelated and purely
> > cosmetic:
> > >
> > > [49569.968486] intel-vbtn INT33D6:00: unknown event index 0xcd
> >
> > I'm quite confident that this is not the event that woke you up, but can
> you
> > paste more lines from dmesg around this for more context?
>
> Here's a complete suspend/resume:

[cut]

> [ 442.460866] Restarting tasks ... done.
> [ 442.552172] thermal thermal_zone11: failed to read out thermal zone (-5)
> [ 442.553302] [drm] RC6 on
> [ 442.630048] IPv6: ADDRCONF(NETDEV_UP): wlp60s0: link is not ready
> [ 442.690004] intel-vbtn INT33D6:00: unknown event index 0xcd

Well, so this happens when user space runs again, so it's not the same event.

Most likely there is one more event coming from the EC in addition to the first wakeup one and sure enough it is not in the driver's keymap. Again, I need to have a look into the ACPI tables of this machine to see what may be going on here.

[rjw (rjw-linux-kernel-bugs)]

Let's set the lid thing aside.

I will ask you to test a couple of things, but first, after doing this:

# echo enabled > /sys/devices/platform/i8042/serio0/power/wakeup

you should be able to use the keyboard to wake up the machine from suspend-to-idle. Please check if that works for you.

If it works, it is sufficient to do the above once per boot from the init scripts.

[rjw (rjw-linux-kernel-bugs)]

Created attachment 257577
ACPI / EC: Add module parameter to disable system wakeup

If the keyboard wakeup works for you (as per the previous comment), this patch will allow you to go back to the non-functional EC wakeup (then you will have to use the keyboard to wake up the system), for example by doing

# echo Y > /sys/module/acpi/parameters/ec_no_wakeup

as root (the default setting can be restored by writing N to this file).

If the keyboard wakeup works, please try that and attach dmesg after suspend/resume.

[rjw (rjw-linux-kernel-bugs)]

The patch from the previous comment should apply on top of 4.13-rc1 (it will not apply to the plain 4.12 in particular).

[rjw (rjw-linux-kernel-bugs)]

Created attachment 257579
PM: Avoid printing suspend debug messages by default

This, in turn, should make all of the ugly debug stuff go away from dmesg, but of course it won't reduce the churn, just the dmesg noise resulting from it.

[patrik.kullman (patrik.kullman-linux-kernel-bugs)]

(In reply to Rafael J. Wysocki from comment #100)
> Let's set the lid thing aside.
>
> I will ask you to test a couple of things, but first, after doing this:
>
> # echo enabled > /sys/devices/platform/i8042/serio0/power/wakeup
>
> you should be able to use the keyboard to wake up the machine from
> suspend-to-idle. Please check if that works for you.
>
> If it works, it is sufficient to do the above once per boot from the init
> scripts.

Ok, I have tried this and it works.

It was a while ago I compiled my own kernel but I'll have a look at it tonight.

As another route, would it be possible to fix the EC platform bug with a BIOS upgrade if Dell/Intel was informed of this? Would you know who to talk to ?

[rjw (rjw-linux-kernel-bugs)]

(In reply to Patrik Kullman from comment #104)
> (In reply to Rafael J. Wysocki from comment #100)
> > Let's set the lid thing aside.
> >
> > I will ask you to test a couple of things, but first, after doing this:
> >
> > # echo enabled > /sys/devices/platform/i8042/serio0/power/wakeup
> >
> > you should be able to use the keyboard to wake up the machine from
> > suspend-to-idle. Please check if that works for you.
> >
> > If it works, it is sufficient to do the above once per boot from the init
> > scripts.
>
> Ok, I have tried this and it works.

OK

> It was a while ago I compiled my own kernel but I'll have a look at it
> tonight.

Cool, thanks!

> As another route, would it be possible to fix the EC platform bug with a
> BIOS upgrade if Dell/Intel was informed of this? Would you know who to talk
> to ?

I'd love that to happen and Dell is aware of the problem already.

[Joseph Salisbury (jsalisbury)]

I built a 4.13-rc1 based test kernel with the patch. It can be downloaded from:

http://kernel.ubuntu.com/~jsalisbury/lp1705099/

[patrik.kullman (patrik.kullman-linux-kernel-bugs)]

Created attachment 257591
dmesg with 4.13-rc1 + no_ec_wakeup

I got the nice help from jsalisbury from #ubuntu-kernel IRC to build me a kernel with the patch (https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1705099) and I'm happy to say that it works perfectly!

Attaching the dmesg and the last suspend/resume was with keyboard wake on and no_ec_wakeup=Y.. and it slept through it all! :)

Do as you please with the other patch, but please make the first one go into 4.13-rc2 :) This feels like the "perfect workaround" right now.

[Patrik Kullman (nomego)]

It works beautifully, I've updated the kernel.org bug.

[rjw (rjw-linux-kernel-bugs)]

(In reply to Patrik Kullman from comment #106)
> Created attachment 257591 [details]
> dmesg with 4.13-rc1 + no_ec_wakeup
>
> I got the nice help from jsalisbury from #ubuntu-kernel IRC to build me a
> kernel with the patch
> (https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1705099) and I'm happy
> to say that it works perfectly!
>
> Attaching the dmesg and the last suspend/resume was with keyboard wake on
> and no_ec_wakeup=Y.. and it slept through it all! :)
>
> Do as you please with the other patch, but please make the first one go into
> 4.13-rc2 :) This feels like the "perfect workaround" right now.

Posted as https://patchwork.kernel.org/patch/9850147/ and we'll see what happens.

I may need to make changes to it, so -rc2 may not be a realistic target, but I think the patch is fair enough in principle.

[AxelMKlein (axelmklein-linux-kernel-bugs)]

Hello all,

I own an XPS 13 9365 and I am enjoying the improvements you are bringing to the usability of the "package" Linux-on-XPS-13-9365 every day!
I mostly use the hibernate-feature becaus it's working best.

I just want to thank you guys for the work you are doing.

You are doing a great job and I hope the suspend will also work flawlessly,
although I fear that the s2idle-mode is not as energy-saving as it could be with s3.
As far as I've understood, on Windows also the s2idle is used and not the s3.
Is this correct?
And if yes, do you have an idea why one (Dell, Intel) creates a mobile system that is not capable of using a suspend mode that keeps the battery-drain low?

Thank you in advance for a potential answer and of course again for your great work.

Best regards
Axel

[patrik.kullman (patrik.kullman-linux-kernel-bugs)]

A few last comments,

For anyone finding this bug to adjust these settings, "once per boot from the init scripts" isn't that easy anymore with systemd :)

The cleanest way I found was to install sysfsutils and adding this into /etc/sysfs.conf:

devices/platform/i8042/serio0/power/wakeup = enabled
module/acpi/parameters/ec_no_wakeup = Y

Also, Rafael, what about the unknown events? Should I create a separate bug for those:

intel-vbtn INT33D6:00: unknown event index 0xcd

[rjw (rjw-linux-kernel-bugs)]

(In reply to Patrik Kullman from comment #109)
> A few last comments,
>
> For anyone finding this bug to adjust these settings, "once per boot from
> the init scripts" isn't that easy anymore with systemd :)
>
> The cleanest way I found was to install sysfsutils and adding this into
> /etc/sysfs.conf:
>
> devices/platform/i8042/serio0/power/wakeup = enabled
> module/acpi/parameters/ec_no_wakeup = Y

You could also use a udev rule for that I think.

> Also, Rafael, what about the unknown events? Should I create a separate bug
> for those:
>
> intel-vbtn INT33D6:00: unknown event index 0xcd

That's just a genuinely unknown event, so it's a separate issue, but I guess it's better to send e-mail to the driver maintainer/author in this case. You can CC me too. :-)

[paulepanter (paulepanter-linux-kernel-bugs)]

(In reply to Rafael J. Wysocki from comment #110)
> (In reply to Patrik Kullman from comment #109)
> > A few last comments,
> >
> > For anyone finding this bug to adjust these settings, "once per boot from
> > the init scripts" isn't that easy anymore with systemd :)
> >
> > The cleanest way I found was to install sysfsutils and adding this into
> > /etc/sysfs.conf:
> >
> > devices/platform/i8042/serio0/power/wakeup = enabled
> > module/acpi/parameters/ec_no_wakeup = Y
>
> You could also use a udev rule for that I think.

Does passing `acpi.ec_no_wakeup=1` on the Linux command line work?

[…]

[superm1 (superm1-linux-kernel-bugs)]

> intel-vbtn INT33D6:00: unknown event index 0xcd

Please CC me on the bugs you file regarding this too. I think there are some events missing related to when you switch tablet/regular mode and this most likely corresponds to one of them.

> And if yes, do you have an idea why one (Dell, Intel) creates a mobile system
> that is not capable of using a suspend mode that keeps the battery-drain low?

Theoretically S2I/Modern Standby is supposed to be similar battery consumption as S3 when all the (moving) parts are properly optimized.

[rjw (rjw-linux-kernel-bugs)]

(In reply to Paul Menzel from comment #111)
> (In reply to Rafael J. Wysocki from comment #110)
>
> Does passing `acpi.ec_no_wakeup=1` on the Linux command line work?
>
> […]

Yes, it should.

[Patrik Kullman (nomego)]

Is CONFIG_ALLOW_DEV_COREDUMP = Y in this kernel ?
Trying to supply info to https://bugzilla.kernel.org/show_bug.cgi?id=196411 now.. :)

Otherwise would you mind rolling a new one with this option on ? :)

[Joseph Salisbury (jsalisbury)]

Yes tha value is set:

config.common.ubuntu:CONFIG_ALLOW_DEV_COREDUMP=y

[Patrik Kullman (nomego)]

Oh ok thanks.

[patrik.kullman (patrik.kullman-linux-kernel-bugs)]

Rafael, I can't really follow patchwork and the git web client.
Did it make it into rc2? I don't see any objections?

[rjw (rjw-linux-kernel-bugs)]

No, it is in linux-next right now and I'm going to push it for -rc3.

[patrik.kullman (patrik.kullman-linux-kernel-bugs)]

Oh ok great!

[Patrik Kullman (nomego)]

Any idea why I can't get a device coredump? (Discussion in https://bugzilla.kernel.org/show_bug.cgi?id=196411)

[Patrik Kullman (nomego)]

Thanks for the help, as for a third kernel bug being hunted down (https://bugzilla.kernel.org/show_bug.cgi?id=196415), would it be possible to get a 4.13-rc3 kernel with CONFIG_ACPI_DEBUG turned on?

The patch in this case has been included in 4.13-rc3.

Should I create a new bug ?

[Joseph Salisbury (jsalisbury)]

Sorry for the delay. It might be best to open a new bug, that way they can be tracked separately.

[samshepard90 (samshepard90-linux-kernel-bugs)]

(In reply to Patrik Kullman from comment #116)
> Oh ok great!

Dear Patrik and Rafael. Thank you both for your efforts on solving this issue. As a linux user who is not as tech savvy would it be possible to get an idiots explanation of how to apply this fix? Is it just a case of updating the kernel? I have updated the kernel to version 4.13 on ubuntu 16.04 and still the issue persists.

[superm1 (superm1-linux-kernel-bugs)]

@Sam,

The behavior to pick S2I over S3 by default didn't land in 4.13, it's going to be in 4.14 however (unless reverted). What you'll want to do is change it on your system like this:

echo "s2idle" | sudo tee /sys/power/mem_sleep

That will default to s2idle for your running session. If that works well for you you can add to your kernel command line mem_sleep_default=s2idle. You can do this by modifying GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub and running update-grub.

[samshepard90 (samshepard90-linux-kernel-bugs)]

(In reply to Mario Limonciello from comment #118)
> @Sam,
>
> The behavior to pick S2I over S3 by default didn't land in 4.13, it's going
> to be in 4.14 however (unless reverted). What you'll want to do is change
> it on your system like this:
>
> echo "s2idle" | sudo tee /sys/power/mem_sleep
>
> That will default to s2idle for your running session. If that works well
> for you you can add to your kernel command line mem_sleep_default=s2idle.
> You can do this by modifying GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub
> and running update-grub.

Thanks Mario! The laptop is now waking automatically on opening. I have successfully added the above command to my kernel command line.

I was just struggling with understanding the instructions here (https://liveandletlearn.net/post/dell-xps13-9365-dual-boot/) regarding installing sysfsutils and how to add the mentioned file to my system.

I assume that these additional steps are no longer necessary- is there anything else I need to do other than running 4.13.2 and having added that command line to my kernel command line? (for example the last two commands- # Enable key press to wakeup (devices/platform/i8042/serio0/power/wakeup = enabled) and # Disable heartbeat wakeups during suspend (module/acpi/parameters/ec_no_wakeup = Y)?

[patrik.kullman (patrik.kullman-linux-kernel-bugs)]

Well that blog post refers to my comment https://bugzilla.kernel.org/show_bug.cgi?id=192591#c109 and I would say rather follow those instructions than sending boot params in my opinion since they're less invasive.

The other two options will save dramatically increase battery time since they will prevent micro wakeups every 1-2 seconds.

[AxelMKlein (axelmklein-linux-kernel-bugs)]

Having included

- 'mem_sleep_default=s2idle' into the GRUB_CMDLINE_LINUX_DEFAULT,

- installed sysfsutils and
- included 'devices/platform/i8042/serio0/power/wakeup = enabled' and
  'module/acpi/parameters/ec_no_wakeup = Y'

  into the /etc/sysfs.conf of my Dell XPS-13 9365,

I experience pretty accurately a battery drain of 5% per hour with these settings in suspend.
Does this meet other user's experience?

I am curious whether this is the 'performance' we can expect.

Best regards
Axel

[AxelMKlein (axelmklein-linux-kernel-bugs)]

Hi,

compared to this 5 per hour battery drain while in sleep, I've found a maximum battery drain of 2-3% independent of sleep time when the 9365 runs Windows.

I assume they have a mechanism in place that changes the state to 'hibernate' after a certain time. For example when I wake up the 9365 after half an hour from sleep, it comes back very quickly, and when I wake it up after one hour from sleep it goes through a booting process that takes much more time.

If this is true, I am asking whether we could have a similar behavior running Linux on this machines.
To me such a procedure looks very sensible: Having it waking up very quickly within half an hour or so and limiting the battery drain to a couple of percents.
This is exactly what I would wish.

So I am asking whether we could have a similar behavior running Linux on this machines?

By the way: Is this the right location to ask such a question?

Best regards

[superm1 (superm1-linux-kernel-bugs)]

@axel,

This comparison matches the performance behavior I've been seeing on some other models too. I'd expect more improvements to be done in the future so that the drain is "lower" and these two more closely match.

As for your question, I believe that functionality is possible to achieve too.
That functionality is best designated to userspace however (such as systemd performing a hybrid suspend when it detects s2idle). It would be best to bring that up with the systemd mailing lists.

[AxelMKlein (axelmklein-linux-kernel-bugs)]

@Mario

Thank you, Mario!

I filed this into the systemd-devel mailinglist.

[Esokrarkose (esokrarkose-linux-kernel-bugs)]

Created attachment 274135
Benchmark of kernel 4.9

I have used the 4.9x kernel series for a long time and patched the kernel, back then the patch was VERY effective as you can see (25ms for acpi_pm_finish).

[Esokrarkose (esokrarkose-linux-kernel-bugs)]

Sorry, posted to the wrong bug :-(.

[ucelsanicin (ucelsanicin-linux-kernel-bugs)]

file=0xe346e0 "/home/vries/gdb_versions/devel/src/gdb/infrun.c", line=6384,
    fmt=0xe34269 "%s: Assertion `%s' failed.", ap=0x7fffffffcb98) https://www.webb-dev.co.uk/sports/gym-during-covid/
    at /home/vries/gdb_versions/devel/src/gdb/utils.c:414
#4 0x0000000000a9c2e2 in internal_verror ( http://www.compilatori.com/health/covid-and-tech/
    file=0xe346e0 "/home/vries/gdb_versions/devel/src/gdb/infrun.c", line=6384,
    fmt=0xe34269 "%s: Assertion `%s' failed.", ap=0x7fffffffcb98) http://www.acpirateradio.co.uk/health/covid-and-tech/
    at /home/vries/gdb_versions/devel/src/gdb/utils.c:439
#5 0x0000000000d39725 in internal_error ( http://www.logoarts.co.uk/health/covid-and-tech/
    file=0xe346e0 "/home/vries/gdb_versions/devel/src/gdb/infrun.c", line=6384,
    fmt=0xe34269 "%s: Assertion `%s' failed.")
    at /home/vries/gdb_versions/devel/src/gdbsupport/errors.cc:55 http://www.slipstone.co.uk/health/covid-and-tech/
#6 0x000000000074b047 in process_event_stop_test (ecs=0x7fffffffd270)
    at /home/vries/gdb_versions/devel/src/gdb/infrun.c:6383 http://embermanchester.uk/health/covid-and-tech/
#7 0x000000000074ad3b in handle_signal_stop (ecs=0x7fffffffd270)
    at /home/vries/gdb_versions/devel/src/gdb/infrun.c:6277
#8 0x0000000000749232 in handle_inferior_event (ecs=0x7fffffffd270)
    at /home/vries/gdb_versions/devel/src/gdb/infrun.c:5530 http://connstr.net/health/covid-and-tech/
#9 0x00000000007456e4 in fetch_inferior_event ()
    at /home/vries/gdb_versions/devel/src/gdb/infrun.c:3912
#10 0x000000000072af38 in inferior_event_handler http://joerg.li/health/covid-and-tech/ (event_type=INF_REG_EVENT)
    at /home/vries/gdb_versions/devel/src/gdb/inf-loop.c:42
#11 0x000000000078584c in handle_target_event (error=0, client_data=0x0)
    at /home/vries/gdb_versions/devel/src/gdb/linux-nat.c:4060 http://www.jopspeech.com/health/covid-and-tech/
#12 0x0000000000d3a447 in handle_file_event (file_ptr=0x56c3aa0, ready_mask=1)
    at /home/vries/gdb_versions/devel/src/gdbsupport/event-loop.cc:575
#13 0x0000000000d3a9cf in gdb_wait_for_event (block=0) http://www.wearelondonmade.com/health/covid-and-tech/
    at /home/vries/gdb_versions/devel/src/gdbsupport/event-loop.cc:701
#14 0x0000000000d39859 in gdb_do_one_event ()
    at /home/vries/gdb_versions/devel/src/gdbsupport/event-loop.cc:212 https://waytowhatsnext.com/sports/asian-sports/
#15 0x0000000000a26343 in wait_sync_command_done ()
    at /home/vries/gdb_versions/devel/src/gdb/top.c:526 http://www.iu-bloomington.com/sports/honda-civic/
#16 0x0000000000a263bb in maybe_wait_sync_command_done (was_sync=0)
    at /home/vries/gdb_versions/devel/src/gdb/top.c:543 https://komiya-dental.com/sports/telegram/
#17 0x0000000000a26953 in execute_command (p=0x7fffffffe15d "", from_tty=0)
    at /home/vries/gdb_versions/devel/src/gdb/top.c:670 http://www-look-4.com/health/covid-and-tech/
#18 0x00000000007ae648 in catch_command_errors (
    command=0xa263d4 <execute_command(char const*, int)> , arg=0x7fffffffe15c "n", from_tty=0)