snap refresh slows down computer dramatically

[Alan Pope 🍺🐧🐱 🦄]

I sometimes issue a "snap refresh" to force some applications to update when I know there happen to be updates. When I start a "snap refresh" within moments my laptop becomes almost completely unusable. Videos playing or video meetings will break up, dropping to a low frame rate and have choppy audio.

This is consistent across different snaps, and doesn't seem dependent on how many I have installed. For example, just now I refreshed and had only 5 snaps updating. I have a ThinkPad T450 with i7 CPU and 16GB RAM. It's got a relatively new, relatively fast SSD, but really doesn't feel like it.

Attached is the output of "snap change 1301" - the most recent refresh.

[Alan Pope 🍺🐧🐱 🦄 (popey)]

alan@KinkPad-K450:~$ snap version
snap 2.40
snapd 2.40
series 16
neon 18.04
kernel 4.18.0-24-generic

[Alan Pope 🍺🐧🐱 🦄 (popey)]

dstat output from refreshing one simple small snap

--total-cpu-usage-- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai stl| read writ| recv send| in out | int csw
 51 14 35 0 0| 439k 788k| 0 0 | 25k 34k|2530 4500
  5 1 94 0 0|4096B 4096B|1009B 1310B|4096B 0 |1589 2676
  5 3 91 0 0| 0 0 | 169B 141B| 0 0 |2345 3967
  6 2 93 0 0| 0 4096B| 415B 581B| 0 0 |2087 3106
  6 4 90 0 0| 126k 0 | 21k 40k| 0 0 |2565 5030
  7 3 90 1 0| 359k 0 | 55k 110k| 0 0 |3201 7153
 11 3 86 1 0|1067k 140k| 53k 104k| 432k 0 |3533 7530
  5 3 92 0 0| 453k 0 | 50k 99k| 0 0 |3372 6224
  5 2 92 1 0| 564k 16k| 57k 112k|4096B 0 |2814 5756
  7 3 90 0 0| 530k 0 | 49k 99k|4096B 0 |3129 6787
  6 3 91 0 0| 492k 0 | 53k 111k| 0 0 |3046 6801
  4 3 92 1 0| 346k 64k| 47k 93k| 0 0 |2630 5604
  5 2 92 1 0| 566k 0 | 54k 108k| 0 0 |2947 6685
  5 3 91 0 0| 461k 372k| 55k 112k| 0 0 |3060 6875
  5 2 93 0 0| 480k 0 | 51k 100k| 0 0 |2802 6324
  9 2 88 1 0|3718k 0 | 41k 76k| 16k 0 |4886 7094
 11 3 85 0 0|4718k 64k|2326B 80k| 16k 0 |3318 5053
 18 3 77 1 0| 193k 10M| 612k 20k| 56k 0 |2959 7097
 17 4 78 1 0| 0 13M|3299k 52k| 0 0 |3980 9766
 19 5 75 2 0| 24k 15M|4873k 92k| 24k 0 |4772 13k
 25 3 70 2 0| 32k 28M|2572k 57k| 0 0 |4022 11k
 58 14 27 2 0|7248k 12M|2991B 911B| 12k 0 |5564 16k
 81 14 5 1 0| 733k 12M|1004B 1032B|8192B 0 |4740 8894
 73 11 14 2 0|3753k 17M|1370B 1727B| 0 0 |3806 10k
 47 8 44 1 0|6422k 16M|1504B 1435B|8192B 0 |3287 7180
 72 14 14 1 0| 97k 4924k|2366B 1646B| 16k 0 |3060 12k
 62 13 25 0 0| 36k 0 | 673B 661B| 36k 0 |3243 12k
 46 16 38 1 0| 368k 7624k|2938B 56k| 0 0 |3416 6952
 34 7 58 0 0| 30k 1900k| 26k 9658B|4096B 0 |2389 6781
 30 5 65 0 0| 0 0 | 19k 4345B| 0 0 |2010 4335
 29 5 66 0 0| 0 0 |3185B 2145B| 0 0 |2153 3997
 32 6 62 0 0| 24k 0 |1223B 1032B| 0 0 |2569 4350
 27 4 69 0 0| 0 0 |5905B 3296B| 0 0 |2337 3214
 27 6 68 0 0| 0 72k|1637B 3825B| 0 0 |2096 3721
 27 5 68 0 0| 0 0 | 476B 476B| 0 0 |2338 3155
 29 5 66 0 0| 0 0 |1482B 822B| 0 0 |2130 3711
 28 4 68 0 0| 0 0 | 475B 768B| 0 0 |1833 2927
 33 4 63 0 0| 0 0 |2542B 3052B| 0 0 |1962 3373
 25 5 70 0 0| 0 36k|1904B 1610B| 0 0 |1739 2924
 19 4 77 0 0| 0 0 |1094B 854B| 0 0 |1795 3029
  7 2 90 0 0| 0 0 |1775B 1751B| 0 0 |1642 3146

Seems a lot of write operations (middle section) where the snap was being refreshed.

[John Lenton (chipaca)]

Could you create /etc/systemd/system/snapd.service.d/prio.conf with

[Service]
IOSchedulingClass = best-effort
IOSchedulingPriority = 7

in it, followed by running

systemctl daemon-reload && systemctl restart snapd.service

and seeing if you can reproduce the issue?

If this is what I think it is, it _should_ be handled by the io scheduler, but maybe lowering our priority improves things.

[Alan Pope 🍺🐧🐱 🦄 (popey)]

Thanks John! That dramatically improved things! Previously i couldn't have a youtube video running while a snap refresh was happening. I just tested with your change in place while watching a video. There were 8 snaps to refresh and they did so without adversely affecting the system as far as I could tell. Much better.

Is this something we can incorporate into snapd? I've seen others report this issue, but don't know how widespread it is.

[John Lenton (chipaca)]

Next would be to test what's the sweet spot for the Priority.

We'd have to test that it doesn't bite us when running inside lxd (as in lp:1709536) but that should be fine as it's lowering, not raising, a priority.

Other than that, I don't see why we couldn't include this as a default.

[John Lenton (chipaca)]

can you change the priority to 2, daemon-reload && restart, and report back?

[Jan Koci (cicindel)]

Actually almost any snap operation consumes 2GB of my RAM - problem is, one of my laptops only has 2GB of RAM total (so it starts to swap like crazy).