snapd

snap command wrongly stacked up the messages while there are CJK characters in the line.

Bug #1952500 reported by Tao Wang
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
snapd
New
Undecided
Unassigned

Bug Description

When install/download a snap using `snap install ...`, there will be a progress bar, it is correctly display while in English locale, however, it will stacked up in CJK locale, such as `zh-cn`.

Here is what happened in `zh-cn` locale:

```shell
➜ sudo snap install --edge icalingua
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge"
下载 snap "icalingua" (69)，来自频道 "edge" 0% 0B/
下载 snap "icalingua" (69)，来自频道 "edge" 0% 0B/
下载 snap "icalingua" (69)，来自频道 "edge" 0% 0B/
下载 snap "icalingua" (69)，来自频道 "edge" 0% 0B/
下载 snap "icalingua" (69)，来自频道 "edge" 0% 0B/
下载 snap "icalingua" (69)，来自频道 "edge" 0% 0B/
下载 snap "icalingua" (69)，来自频道 "edge" 0% 0B/
下载 snap "icalingua" (69)，来自频道 "edge" 0% 0B/
下载 snap "icalingua" (69)，来自频道 "edge" 0% 0B/
下载 snap "icalingua" (69)，来自频道 "edge" 0% 0B/
下载 snap "icalingua" (69)，来自频道 "edge" 0% 249kB/
下载 snap "icalingua" (69)，来自频道 "edge" 0% 228kB/
下载 snap "icalingua" (69)，来自频道 "edge" 0% 210kB/
下载 snap "icalingua" (69)，来自频道 "edge" 0% 391kB/
下载 snap "icalingua" (69)，来自频道 "edge" 1% 547kB/
下载 snap "icalingua" (69)，来自频道 "edge" 1% 684kB/
下载 snap "icalingua" (69)，来自频道 "edge" 1% 805kB/
下载 snap "icalingua" (69)，来自频道 "edge" 1% 912kB/
下载 snap "icalingua" (69)，来自频道 "edge" 2% 1.15MB/
下载 snap "icalingua" (69)，来自频道 "edge" 2% 1.37MB/
下载 snap "icalingua" (69)，来自频道 "edge" 2% 1.56MB/
下载 snap "icalingua" (69)，来自频道 "edge" 3% 1.74MB/
下载 snap "icalingua" (69)，来自频道 "edge" 3% 1.90MB/
下载 snap "icalingua" (69)，来自频道 "edge" 4% 2.05MB/
下载 snap "icalingua" (69)，来自频道 "edge" 4% 2.19MB/
下载 snap "icalingua" (69)，来自频道 "edge" 5% 2.32MB/
下载 snap "icalingua" (69)，来自频道 "edge" 5% 2.43MB/
下载 snap "icalingua" (69)，来自频道 "edge" 5% 2.54MB/
下载 snap "icalingua" (69)，来自频道 "edge" 6% 2.64MB/
下载 snap "icalingua" (69)，来自频道 "edge" 6% 2.74MB/
下载 snap "icalingua" (69)，来自频道 "edge" 7% 2.82MB/
下载 snap "icalingua" (69)，来自频道 "edge" 7% 2.91MB/
下载 snap "icalingua" (69)，来自频道 "edge" 7% 2.99MB/
下载 snap "icalingua" (69)，来自频道 "edge" 8% 3.06MB/
下载 snap "icalingua" (69)，来自频道 "edge" 8% 3.13MB/
下载 snap "icalingua" (69)，来自频道 "edge" 8% 3.12MB/
下载 snap "icalingua" (69)，来自频道 "edge" 9% 3.18MB/
下载 snap "icalingua" (69)，来自频道 "edge" 9% 3.31MB/
下载 snap "icalingua" (69)，来自频道 "edge" 10% 3.30MB/
下载 snap "icalingua" (69)，来自频道 "edge" 10% 3.35MB/

...

下载 snap "icalingua" (69)，来自频道 "edge" 95% 4.83MB/
下载 snap "icalingua" (69)，来自频道 "edge" 96% 4.83MB/
下载 snap "icalingua" (69)，来自频道 "edge" 96% 4.83MB/
下载 snap "icalingua" (69)，来自频道 "edge" 96% 4.84MB/
下载 snap "icalingua" (69)，来自频道 "edge" 97% 4.84MB/
下载 snap "icalingua" (69)，来自频道 "edge" 97% 4.84MB/
下载 snap "icalingua" (69)，来自频道 "edge" 98% 4.84MB/
下载 snap "icalingua" (69)，来自频道 "edge" 98% 4.84MB/
下载 snap "icalingua" (69)，来自频道 "edge" 98% 4.85MB/
下载 snap "icalingua" (69)，来自频道 "edge" 99% 4.85MB/
下载 snap "icalingua" (69)，来自频道 "edge" 99% 4.85MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.85MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.85MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.83MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.82MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.80MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.78MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.76MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.75MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.73MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.71MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.70MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.68MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.66MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.65MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.63MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.62MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.60MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.58MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.57MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.55MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.54MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.52MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.51MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.49MB/
下载 snap "icalingua" (69)，来自频道 "edge" 100% 4.48MB/
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
获取并检查 snap "icalingua" (69) 的断言
将 icalingua:icon-themes 连接到 gtk-common-themes:icon-themes
将 icalingua:browser-support 连接到 core:browser-support
icalingua (edge) 2.4.4-Deus-non-vult-12-g5db99fc from Tao Wang (twang2218) installed
```

The reason of it stacked up the messages is that CJK characters are **twice width** of english characters. So, when the `snap` command calculate number of the filling characters, it count the CJK characters as 1 time width, which should be 2.

For example,

`获取并检查 snap "icalingua" (69) 的断言`

The about string is count as width `31`, however, it contains **8 CJK characters**, which should be count as **16** width, so the total width of the string is `39`.

It not only calculate wrongly on the `snap install`, it also stacked the messages on `snap remove ...`.

```shell
➜ sudo snap remove icalingua
将 icalingua:network 从 core:network 断开
将 icalingua:network 从 core:network 断开
将 icalingua:network 从 core:network 断开
将 icalingua:network 从 core:network 断开
将 icalingua:network 从 core:network 断开
将 icalingua:network 从 core:network 断开
将 icalingua:gnome-3-28-1804 从 gnome-3-28-1804:gnome-3-28-1804 断开
将 icalingua:gnome-3-28-1804 从 gnome-3-28-1804:gnome-3-28-1804 断开
将 icalingua:gnome-3-28-1804 从 gnome-3-28-1804:gnome-3-28-1804 断开
将 icalingua:gnome-3-28-1804 从 gnome-3-28-1804:gnome-3-28-1804 断开
将 icalingua:gnome-3-28-1804 从 gnome-3-28-1804:gnome-3-28-1804 断开
将 icalingua:opengl 从 core:opengl 断开
将 icalingua:opengl 从 core:opengl 断开
将 icalingua:opengl 从 core:opengl 断开
将 icalingua:opengl 从 core:opengl 断开
将 icalingua:opengl 从 core:opengl 断开
将 icalingua:opengl 从 core:opengl 断开
将 icalingua:opengl 从 core:opengl 断开
将 icalingua:opengl 从 core:opengl 断开
将 icalingua:opengl 从 core:opengl 断开
将 icalingua:opengl 从 core:opengl 断开
将 icalingua:opengl 从 core:opengl 断开
将 icalingua:opengl 从 core:opengl 断开
将 icalingua:opengl 从 core:opengl 断开
将 icalingua:desktop-legacy 从 core:desktop-legacy 断开
将 icalingua:desktop-legacy 从 core:desktop-legacy 断开
将 icalingua:desktop-legacy 从 core:desktop-legacy 断开
将 icalingua:desktop-legacy 从 core:desktop-legacy 断开
将 icalingua:desktop-legacy 从 core:desktop-legacy 断开
将 icalingua:unity7 从 core:unity7 断开
将 icalingua:unity7 从 core:unity7 断开
将 icalingua:unity7 从 core:unity7 断开
将 icalingua:unity7 从 core:unity7 断开
将 icalingua:wayland 从 core:wayland 断开
将 icalingua:wayland 从 core:wayland 断开
将 icalingua:wayland 从 core:wayland 断开
将 icalingua:wayland 从 core:wayland 断开
将 icalingua:audio-playback 从 core:audio-playback 断开
将 icalingua:audio-playback 从 core:audio-playback 断开
将 icalingua:audio-playback 从 core:audio-playback 断开
将 icalingua:audio-playback 从 core:audio-playback 断开
将 icalingua:gtk-3-themes 从 gtk-common-themes:gtk-3-themes 断开
将 icalingua:gtk-3-themes 从 gtk-common-themes:gtk-3-themes 断开
将 icalingua:gtk-3-themes 从 gtk-common-themes:gtk-3-themes 断开
将 icalingua:sound-themes 从 gtk-common-themes:sound-themes 断开
将 icalingua:sound-themes 从 gtk-common-themes:sound-themes 断开
将 icalingua:sound-themes 从 gtk-common-themes:sound-themes 断开
将 icalingua:desktop 从 core:desktop 断开
将 icalingua:desktop 从 core:desktop 断开
将 icalingua:desktop 从 core:desktop 断开
将 icalingua:x11 从 core:x11 断开
将 icalingua:x11 从 core:x11 断开
将 icalingua:x11 从 core:x11 断开
将 icalingua:x11 从 core:x11 断开
将 icalingua:icon-themes 从 gtk-common-themes:icon-themes 断开
将 icalingua:icon-themes 从 gtk-common-themes:icon-themes 断开
将 icalingua:browser-support 从 core:browser-support 断开
将 icalingua:browser-support 从 core:browser-support 断开
将 icalingua:browser-support 从 core:browser-support 断开
将 icalingua:home 从 core:home 断开
将 icalingua:home 从 core:home 断开
将 icalingua:gsettings 从 core:gsettings 断开
将 icalingua:gsettings 从 core:gsettings 断开
icalingua removed
```

See original description
Revision history for this message
Tao Wang (twang2218) wrote :
Revision history for this message
Tao Wang (twang2218) wrote :
description: updated
Revision history for this message
Tao Wang (twang2218) wrote :

I checked the 'progress/ansimeter.go', it is used the number of `rune`, rather than calculate the real width.

https://github.com/snapcore/snapd/blob/master/progress/ansimeter.go#L150-L160

 rpercent := []rune(percent)
 rspeed := []rune(speed)
 rtimeleft := []rune(timeleft)
 msg := make([]rune, 0, col)
 // XXX: assuming terminal can display `col` number of runes
 msg = append(msg, norm(col-len(rpercent)-len(rspeed)-len(rtimeleft), p.label)...)
 msg = append(msg, rpercent...)
 msg = append(msg, rspeed...)
 msg = append(msg, rtimeleft...)
 i := int(current * float64(col) / p.total)
 fmt.Fprint(stdout, "\r", enterReverseMode, string(msg[:i]), exitAttributeMode, string(msg[i:]))

As a reference, here is how project 'schollz/progressbar' calculating the width:

https://github.com/schollz/progressbar/blob/master/progressbar.go#L647-L651

```go
 // get the amount of runes in the string instead of the
 // character count of the string, as some runes span multiple characters.
 // see https://stackoverflow.com/a/12668840/2733724
 stringWidth := runewidth.StringWidth(cleanString)
 return stringWidth
```

it use `github.com/mattn/go-runewidth` to calculate the width.

Revision history for this message
Maciej Borzecki (maciek-borzecki) wrote :

Thank you for reporting this. Snapd already works on runes (which are supposed to be composed of one or more bytes, depending on which utf-8 code point it is), but perhaps there is a too simplistic assumption that the terminal can display as many runes as there are columns. If I understand the problem correctly, some runes are wider than a single column, is this what is happening?

Revision history for this message
Tao Wang (twang2218) wrote :

> If I understand the problem correctly, some runes are wider than a single column, is this what is happening?

Yes, many characters in Chinese/Japanese/Korean languages are twice width than alphabet.

Here is how `mattn/go-runewidth` implements such calculation:

https://github.com/mattn/go-runewidth/blob/master/runewidth.go

-----
// RuneWidth returns the number of cells in r.
// See http://www.unicode.org/reports/tr11/
func (c *Condition) RuneWidth(r rune) int {
 if r < 0 || r > 0x10FFFF {
  return 0
 }
 if len(c.combinedLut) > 0 {
  return int(c.combinedLut[r>>1]>>(uint(r&1)*4)) & 3
 }
 // optimized version, verified by TestRuneWidthChecksums()
 if !c.EastAsianWidth {
  switch {
  case r < 0x20:
   return 0
  case (r >= 0x7F && r <= 0x9F) || r == 0xAD: // nonprint
   return 0
  case r < 0x300:
   return 1
  case inTable(r, narrow):
   return 1
  case inTables(r, nonprint, combining):
   return 0
  case inTable(r, doublewidth):
   return 2
  default:
   return 1
  }
 } else {
  switch {
  case inTables(r, nonprint, combining):
   return 0
  case inTable(r, narrow):
   return 1
  case inTables(r, ambiguous, doublewidth):
   return 2
  case !c.StrictEmojiNeutral && inTables(r, ambiguous, emoji, narrow):
   return 2
  default:
   return 1
  }
 }
}
------

As described in the unicode document, http://www.unicode.org/reports/tr11/, most characters in CJK are **Fullwidth**, and those alphabets in language like English are **Halfwidth**. Fullwidth is double width than Halfwidth.

'schollz/progressbar' had similiar issue before, and fixed in this issue: https://github.com/schollz/progressbar/issues/61

Revision history for this message
叮当 (dingdang) wrote :

Any progress? It has been 2 years since last comment.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.