Account creation during first boot of an OEM image needs to scale based on resolution

[Jamie Chang]

[Impact]
The ubiquity window size display in HiDPI screens is small.

The new gSettings to scale GTK applications (and their corresponding settings in System Settings) work great, but this does not help for the account creation (first boot) for users buying a pre-install system.

We would need changes in Ubiquity to:

 * Probe the monitor to get resolution information
 * Re-scale the ubiquity window using the corresponding gsetting (com.ubuntu.user-interface scale-factor) based on resolution

We would need the changes to be backported to Trusty as well.

[Test Case]
1. Install image on a system with HiDPI panel
2. Check the ubiquity window size displayed on screen

Expected Result:
The window size should be reasonable for user to read the content.

Actual Result:
The window size is small and user is hard to read the content.

[Ara Pulido (ara)]

In order to be able to track this better, we would need a more technical bug, filed against a correct package and a possible a test case in stock Ubuntu.

Thanks,
Ara.

[Woodrow Shen (woodrow-shen)]

For 4K dispaly, I modify gsd-background-manager.c of unity-setting-daemon to dump the resolution of gdk screen and window from g_log, and found that when gdk window got the resolution of 3840x2160, gdk screen still got the resolution of 1920x1080.

[Phillip Susi (psusi)]

This looks like a bug in Xwindows or a hardware detection issue rather than any new design needed. Fonts are already scaled by the correct amount so they appear the correct size on a screen. In other words, a 12 point font is 12 / 72nds of an inch tall no matter if you are in 640 x 480 or 1800 x 1600, at least provided that X can detect the physical size of the monitor from its EDID correctly.

[Shih-Yuan Lee (fourdollars)]

There is a criterion to detect HiDPI for gtk/gnome based UI.

"GNOME currently enables hi-dpi support when the screen resolution is at least 192 dpi and the screen height (in device pixels) is at least 1200." excerpt from https://wiki.gnome.org/HowDoI/HiDpi.

I made a little script at https://github.com/fourdollars/scripts/blob/master/ubuntu/desktop-auto-scaliing.py able to be referred to.

[Phillip Susi (psusi)]

HiDPI is for doubling things like bitmaps, whose size is fixed in pixels. In that image I don't really see any bitmaps that would benefit from this, only text, and text is always scaled to the right size. Therefore, it looks like the problem is just that X is not detecting the DPI of the monitor correctly.

[Shih-Yuan Lee (fourdollars)]

There are some additional functions that those bitmaps relative operations should follow up.

<quote>
Hi-DPI for developers

GTK+ makes hi-dpi work transparently in most places, but sometimes it may be necessary to do some extra work for best results. Here are some APIs to learn about the scale factor:

gdk_screen_get_monitor_scale_factor
gdk_window_get_scale_factor
gtk_widget_get_scale_factor

Similarly, Clutter tries to automatically handle the scaling factor on each drawing surface, but it provides explicit API to retrieve and set the scaling factor:

ClutterSettings:window-scaling-factor
clutter_canvas_set_scale_factor
clutter_canvas_get_scale_factor
</quote>

Excerpt from https://wiki.gnome.org/HowDoI/HiDpi.

[Shih-Yuan Lee (fourdollars)]

I found some information in unity-settings-daemon-14.04.0+14.04.20140606/plugins/xsettings/gsd-xsettings-manager.c.

/* As we cannot rely on the X server giving us good DPI information, and
 * that we don't want multi-monitor screens to have different DPIs (thus
 * different text sizes), we'll hard-code the value of the DPI
 *
 * See also:
 * https://bugzilla.novell.com/show_bug.cgi?id=217790•
 * https://bugzilla.gnome.org/show_bug.cgi?id=643704
 *
 * http://lists.fedoraproject.org/pipermail/devel/2011-October/157671.html
 * Why EDID is not trustworthy for DPI
 * Adam Jackson ajax at redhat.com
 * Tue Oct 4 17:54:57 UTC 2011
 *
 * Previous message: GNOME 3 - font point sizes now scaled?
 * Next message: Why EDID is not trustworthy for DPI
 * Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
 *
 * On Tue, 2011-10-04 at 11:46 -0400, Kaleb S. KEITHLEY wrote:
 *
 * > Grovelling around in the F15 xorg-server sources and reviewing the Xorg
 * > log file on my F15 box, I see, with _modern hardware_ at least, that we
 * > do have the monitor geometry available from DDC or EDIC, and obviously
 * > it is trivial to compute the actual, correct DPI for each screen.
 *
 * I am clearly going to have to explain this one more time, forever.
 * Let's see if I can't write it authoritatively once and simply answer
 * with a URL from here out. (As always, use of the second person "you"
 * herein is plural, not singular.)
 *
 * EDID does not reliably give you the size of the display.
 *
 * Base EDID has at least two different places where you can give a
 * physical size (before considering extensions that aren't widely deployed
 * so whatever). The first is a global property, measured in centimeters,
 * of the physical size of the glass. The second is attached to your (zero
 * or more) detailed timing specifications, and reflects the size of the
 * mode, in millimeters.
 *
 * So, how does this screw you?
 *
 * a) Glass size is too coarse. On a large display that cm roundoff isn't
 * a big deal, but on subnotebooks it's a different game. The 11" MBA is
 * 25.68x14.44 cm, so that gives you a range of 52.54-54.64 dpcm horizontal
 * and 51.20-54.86 dpcm vertical (133.4-138.8 dpi h and 130.0-139.3 dpi v).
 * Which is optimistic, because that's doing the math forward from knowing
 * the actual size, and you as the EDID parser can't know which way the
 * manufacturer rounded.
 *
 * b) Glass size need not be non-zero. This is in fact the usual case for
 * projectors, which don't have a fixed display size since it's a function
 * of how far away the wall is from the lens.
 *
 * c) Glass size could be partially non-zero. Yes, really. EDID 1.4
 * defines a method of using these two bytes to encode aspect ratio, where
 * if vertical size is 0 then the aspect ratio is computed as (horizontal
 * value + 99) / 100 in portrait mode (and the obvious reverse thing if
 * horizontal is zero). Admittedly, unlike every other item in this list,
 * I've never seen this in the wild. But it's legal.
 *
 * d) Glass size could be a direct encoding of the aspect ratio. Base EDID
 * doesn't condone this behaviour, but the CEA spec (to which ...

[Stéphane Verdy (sverdy)]

Some feedback from bregma:
"I imagine it's just a matter of writing a script to query the X server (xrandr) and calculating some reasonable scaling factor based on the pixel density, then setting the gsettings value
<bregma> maybe a half-days work for someone who knows Python
<bregma> StephaneVerdy, that bug has some good info on how to write the script (and a link to an example), although in Unity we use a more fine-grained scaling control and allow different scaling on different monitors (probably not an issue for OEM setup)
<bregma> the best bet is to use the GTK devioce info because they work around some broken EDIDs better than X does

[Shih-Yuan Lee (fourdollars)]

A patch for unity-settings-daemon.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "hidpi_display.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Stéphane Graber (stgraber)]

So for the ubiquity task, ubiquity calls unity-settings-daemon when running under ubiquity-dm (as is the case for the standalone installer and for OEM config), so once the unity-settings-daemon issue is resolved, I'd expect the UI to just scale as expected.

As such, marking the ubiquity tasks invalid.

[Shih-Yuan Lee (fourdollars)]

I found we still need to modify ubiquity for the HiDPI support.

[Bruce.Ma (bruce.ma)]

This issue could be reproduced with ubuntu-14.04-desktop-amd64.iso on Mystique-1-3(HiDPI laptop).
[ Test Case]
 * Use usb-creator-gtk to burn ubuntu-14.04-desktop-amd64.iso to USB stick.
 * Use that stick to install Ubuntu.
 * The UI window is small.

[Sebastien Bacher (seb128)]

Should ubuntu-sponsors be subscribed to review those u-s-d changes?

Shih-Yuan, could you merge propose the unity-settings-daemon against the upstream project on launchpad? That would help to get it reviewed

[Sebastien Bacher (seb128)]

I've not looked to the logic behind hidpi scaling in a bit, but iirc unity-settings-daemon doesn't apply the factor to the session.

In GNOME g-s-d does it, but the unity case is more complex since we have different factors (gtk being an int, unity being a float, ...) so we let unity do the magic.

Of course in the oem installer mode unity is not running so nothing is setting the xsettings that would make gtk do the hidpi transformations...

@Shih-Yuan, your patch add support of the unity scaling to u-s-d, but is that really helping in the case where unity is not running? We should probably just make u-s-d apply the gtk factor if unity is not there, that should work right?

[Shih-Yuan Lee (fourdollars)]

I found there are some commits from the upstream and I am trying to merge those commits with my patch.
However the work is not finished yet, so I put it in lp:~fourdollars/unity-settings-daemon/add-hidpi-support temporarily.

[Mathieu Trudel-Lapierre (cyphermox)]

Shih-Yuan; the "ubiquity_hidpi.patch" patch you attached doesn't seem to be directly related to HiDPI support. Maybe I'm misreading, but those seem to be nothing more than some of the changes that aren't yet in Trusty.

As for the changes to unity-settings-daemon, I can only let you finish those since there seemed to be some logic missing to properly calculate the scaling factor. Have you considered Sebastien's option of applying the gtk scaling factor directly from u-s-d if we can't detect unity in the session?

I'll make the ubiquity task Incomplete for now; I'm not saying we can't apply the changes in your patch, but I just couldn't see the relation with HiDPI scaling.

[Bruce.Ma (bruce.ma)]

I already merged the hidpi_display.patch to add-hidpi-support and put it to
lp:~bruce.ma/vivid/unity-settings-daemon.lp1382291

[Bruce.Ma (bruce.ma)]

I already merged the hidpi_display.patch to ubiquity and put it to
lp:~bruce.ma/vivid/ubiquity.lp1382291

[Shih-Yuan Lee (fourdollars)]

Regarding comment #18, the HiDPI support is in the xsettings plugin of u-s-d/g-s-d, the only thing that ubiquity needs to do is waiting for the xsettings plugin before ubiquity invokes any GTK/GDK function.

[Shih-Yuan Lee (fourdollars)]

However the original mechanism for the background image doesn't support HiDPI well, so some part of my patch is to use feh to fix the background image problem.

[Bruce.Ma (bruce.ma)]

I merged Shih-Yuan's patch to trusy and will test it on Mystique-2-3.

[Bruce.Ma (bruce.ma)]

I merged 4$'s patch to unity-settings-daemon and ubiquity of trusty and tested it on Mystique-2-3 with kittyhawk2-trusty-amd64-20150127-0.
The window was scaled correctly during OOBE, but the background was empty.

Detail:
https://bugs.launchpad.net/kittyhawk/kittyhawk-2.0/+bug/1389940/comments/10

[Bruce.Ma (bruce.ma)]

I discussed with Shih-Yuan abouth the background issue, he advised to install the feh package and try again.
I will test this solution.

[Bruce.Ma (bruce.ma)]

After the "feh" package was installed, the windows was scaled correctly during P1, P2, OOBE stage.
But the login window was scaled down incorrectly.
Detail: https://bugs.launchpad.net/kittyhawk/kittyhawk-2.0/+bug/1389940 #14-#16

[Shih-Yuan Lee (fourdollars)]

@Bruce,

Bug #1286878 needs to be fixed with this issue.

[Bruce.Ma (bruce.ma)]

I tested the patch that was uploaded by Shih-Yuan for Bug #1286878, now the login window could be scaled correctly on hidpi display.
https://bugs.launchpad.net/ubuntu/+source/unity-greeter/+bug/1286878/comments/12

[Bruce.Ma (bruce.ma)]

Merged Shih-Yuan's patch to the packages of trusty. And uploaded the code to lp.

$ bzr push lp:~oem-solutions-group/kittyhawk/unity-greeter.lp1286878
Created new branch.

$ bzr push lp:~oem-solutions-group/kittyhawk/unity-settings-daemon.lp1382291
Created new branch.

$ bzr push lp:~oem-solutions-group/kittyhawk/ubiquity.lp1382291
Created new branch.

[Phillip Susi (psusi)]

There really is a cancer destroying GNOME... just because you can't auto detect the correct dpi 100% of the time does not mean you give up trying, and then remove the applet to manually set it from the control panel. We really need to fix that.

[Bruce.Ma (bruce.ma)]

Merged Shih-Yuan's patch to newest code from vivid. And uploaded the code to lp.
 $ bzr push lp:~bruce.ma/vivid/unity-settings-daemon.lp1382291
Created new branch.

$ bzr push lp:~bruce.ma/vivid/ubiquity.lp1382291
Created new branch.

[Bruce.Ma (bruce.ma)]

I don't have the permission to propose a merging to lp:unity-settings-daemon and lp:ubiquity .
 Who can help me to do that ?

[Bin Li (binli)]

@Bruce,

 For vivid you need push you own project in lp:~bruce.ma/unity-settings-daemon/support-hidpi-lp1382291 and merge it into lp:unity-settings-daemon.

 For trusty you need based on lp:unity-settings-daemon/14.04 .

[Bruce.Ma (bruce.ma)]

@Bin Li, thank you.
I've uploaded code to lp:~bruce.ma/unity-settings-daemon/support-hidpi-lp1382291 and proposed a merging to lp:unity-settings-daemon.
https://code.launchpad.net/~bruce.ma/unity-settings-daemon/support-hidpi-lp1382291/+merge/249293

I've uploaded code to lp:~bruce.ma/ubiquity/support-hidpi-lp1382291 and proposed a merging to lp:ubiquity.
https://code.launchpad.net/~bruce.ma/ubiquity/support-hidpi-lp1382291/+merge/249292

[Dimitri John Ledkov (xnox)]

Hello,

From this bug report I see issues with ubiquity window itself, but not the background.

In https://code.launchpad.net/~bruce.ma/ubiquity/support-hidpi-lp1382291/+merge/249292 something very strange is proposed for the Ubuntu Desktop:
 - pull in dependency from universe
 - disable usage of unity-settings-daemon / gnome-settings-daemon
 - use above to "fix ubiquity wallpaper"

Yet from screenshots in this bug report, out of all things the wallpaper is rendered correctly.

I'll chat with Desktop team about this, but I'm not happy at the proposed change for ubiquity, as it does not seem to address the bug report in question at all.

[Shih-Yuan Lee (fourdollars)]

@xnox,

If we do nothing for ubiquity-dm, the background issue will be introduced after unity-settings-daemon is patched.
This is why the background issue is not mentioned in the beginning.

 - pulling 'feh' from universe/graphics is bad if we can find out other solution to fix the background issue.
 - disabling the background usage of unity-settings-daemon / gnome-settings-daemon is bad if we can fix the background issue of u-s-d/g-s-d that uses libgnome-desktop to render the background, and, so far as I know, GNOME doesn't maintain this way to render the background for HiDPI. GNOME desktop use GNOME shell service to render the background now so GNOME desktop works fine.
 - using above to "fix ubiquity wallpaper" is bad, but I can not find a better solution so far.

There are also some other issues, such as the abnormal position of indicator applets, the randomly abnormal size changing of the ubiquity window content, and the randomly abnormal position of the ubiquity window.

The window manager that ubiquity-dm used is metacity instead compiz so there are also some issues here for HiDPI.

If possible, I hope Desktop team can put more resources to fix HiDPI issues in ubiquity/unity-greeter/unity-settings-daemon directly.

[Shih-Yuan Lee (fourdollars)]

Since the change of http://bazaar.launchpad.net/~unity-settings-daemon-team/unity-settings-daemon/trunk/revision/4020 and http://bazaar.launchpad.net/~unity-settings-daemon-team/unity-settings-daemon/14.04/revision/4020, autodetecting the scaling factor is disabled, so many issues are hidden.

[Bruce.Ma (bruce.ma)]

This patch need to be updated , then resubmit to upstream.

[Bruce.Ma (bruce.ma)]

Sebastien Bacher pushed a new patch for unity-settings-daemon, it at :
lp:~seb128/unity-settings-daemon/non-unity-sessions-need-scaling

I tested this patch on a hidpi Lenovo laptop with kittyhawk2-trusty-amd64-iso-20150209-1.iso , it worked well, the windows was larger during installation.

[Bruce.Ma (bruce.ma)]

The code have been pushed to lp.
$ bzr push lp:~oem-solutions-group/kittyhawk/unity-settings-daemon_Seb.lp1382291

[Bruce.Ma (bruce.ma)]

But, after applied this patch , the background is incorrect. It became 4 small background.

[Ara Pulido (ara)]

Bruce, can you attach a screenshot, please?

[Sebastien Bacher (seb128)]

The background rendering issue seems a gnome-desktop3 bug, attaching a small testcase

to compile:
$ gcc -o background background.c `pkg-config --cflags --libs glib-2.0 gdk-3.0 gdk-x11-3.0 gio-2.0 gnome-desktop-3.0`

to try in a normal/unity7 session
- close nautilus (nautilus -q) so nothing renders the background
- run GDK_SCALE=2 ./background
- look at the result

[Sebastien Bacher (seb128)]

using cairo_surface_set_device_scale() in gnome_bg_create_surface() makes it work, not sure if that's the right fix though

[Bruce.Ma (bruce.ma)]

4 small background.

[Sebastien Bacher (seb128)]

opened https://bugzilla.gnome.org/show_bug.cgi?id=746044 upstream about it, maybe they can comment on the issue

[Bruce.Ma (bruce.ma)]

During P1, if set "scaling-factor" of "org.gnome.desktop.interface" to 2, then the "Ubuntu Recovery" window will be scaled to duble size. Then the backgroubnd will be separated to 4 small pictures. And if set "scaling-factor" back to 1, then the background will be back to single picture.

Steps:
1. Switch to tty1, execute:
$ export DISPLAY=:0.0 ; gnome-terminal &
2. Back to Ctrl-F7
3. Execute:
$ gsettings set org.gnome.desktop.interface scaling-factor 2

[Bruce.Ma (bruce.ma)]

The background won't be changed after commented "Gdk/WindowScalingFactor" in plugins/xsettings/gsd-xsettings-manager.c of unity-settings-daemon:

 557 static void
 558 xft_settings_set_xsettings (GnomeXSettingsManager *manager,
 559 GnomeXftSettings *settings)
... ...
 568 xsettings_manager_set_string (manager->priv->managers [i], "Xft/HintStyle", settings->hintstyle);
 569 // xsettings_manager_set_int (manager->priv->managers [i], "Gdk/WindowScalingFactor", settings->window_scale);
 570 xsettings_manager_set_int (manager->priv->managers [i], "Gdk/UnscaledDPI", settings->dpi);
 571 xsettings_manager_set_int (manager->priv->managers [i], "Xft/DPI", settings->scaled_dpi);
... ...
 574 }
 575 gnome_settings_profile_end (NULL);
 576 }

[Bruce.Ma (bruce.ma)]

When the installation was completed, the 2x2 background was shown for 1 second after user login to system on unity-greeter. But the background would be became correct background after user the user's desktop was loaded completely.

[Shih-Yuan Lee (fourdollars)]

Regarding comment #49, the background will be repainted by Nautilus.
Trying to execute `nautilus -q` and you may see the tiled background.

[Lars Karlitski (larsu)]

I've attached a patch for gnome-desktop to the upstream bug[1] that fixes this issue for me.

[1] https://bugzilla.gnome.org/show_bug.cgi?id=746044

[Launchpad Janitor (janitor)]

This bug was fixed in the package gnome-desktop3 - 3.14.1-1ubuntu2

---------------
gnome-desktop3 (3.14.1-1ubuntu2) vivid; urgency=medium

  * debian/patches/gnomebg_hidpi_image.patch:
    - gnome_bg_create_surface: always honor device scale, that fixes
      the wallpaper rendering in hidpi config under ubiquity,
      thanks Lars Uebernickel (lp: #1382291)
 -- Sebastien Bacher <email address hidden> Wed, 18 Mar 2015 18:16:55 +0100

[Bruce.Ma (bruce.ma)]

I tested the patch which at #51. The background issue was fixed. And the new code had been uploaded to lp.
$ bzr push lp:~oem-solutions-group/kittyhawk/gnome-desktop3.lp1382291

The upstream bug: https://bugzilla.gnome.org/show_bug.cgi?id=746044 .
The patch was at: https://bugzilla.gnome.org/attachment.cgi?id=299687&action=diff

[Bruce.Ma (bruce.ma)]

I applied these patch to kittyhawk2.iso , the window was
large and complete. And the background was drawn correctly during
installation stage.
But the 4 small background was shown for seconds during login progress.

[Bruce.Ma (bruce.ma)]

I tested vivid daily image vivid-desktop-amd64-20150408.iso on a hi-dpi Lenovo laptop. It was failed, because the "Installation Type" window was too large, some options was not shown.

[Bruce.Ma (bruce.ma)]

2nd window on hi-dpi laptop.

[Bruce.Ma (bruce.ma)]

3rd window on hi-dpi laptop.

[Bruce.Ma (bruce.ma)]

The 4th window on hi-dpi laptop.
This window counldn't be shown completely.

[Bruce.Ma (bruce.ma)]

The 1st window on low-dpi laptop.

[Bruce.Ma (bruce.ma)]

The 2nd window on low-dpi laptop.

[Bruce.Ma (bruce.ma)]

The 3rd window on low-dpi laptop.

[Bruce.Ma (bruce.ma)]

The 4th window on low-dpi laptop.

[Bruce.Ma (bruce.ma)]

The resolution of the low-dpi laptop.

[Bruce.Ma (bruce.ma)]

Merged the code(#39) to newest kittyhawk/unity-setttings-daemon .
The new unity-settings-daemon have been uploaded to lp.

 $ bzr push lp:~oem-solutions-group/kittyhawk/unity-settings-daemon_Seb.lp1382291

[Bruce.Ma (bruce.ma)]

I tested these patch with newest kh2.0 image (kittyhawk2-trusty-amd64-iso-20150508-1.iso).
The UI was scaled correctly during installation.
But the unity-greeter was not shown correctly.

Patches:
lp:~oem-solutions-group/kittyhawk/gnome-desktop3.lp1382291
lp:~oem-solutions-group/kittyhawk/unity-settings-daemon_Seb.lp1382291

[Bruce.Ma (bruce.ma)]

Quote: https://bugs.launchpad.net/ubuntu/+source/unity-greeter/+bug/1286878/comments/19
>>> This bug was fixed in the package unity-settings-daemon - 15.04.1+15.04.20150318-0ubuntu1

That fix is for unity-settings-daemon of vivid , I will try to backport it to trusty.

[Ara Pulido (ara)]

Bruce, any updates on this?

[Bin Li (binli)]

After review the change in comments 19, I found the patch is already on the below.

https://bugs.launchpad.net/ubuntu/+source/unity-greeter/+bug/1286878/comments/19

lp:~oem-solutions-group/kittyhawk/unity-settings-daemon_Seb.lp1382291

[Bin Li (binli)]

Ara,

 I update the patch to fix the issue in #55.

 I set a lower limit resolution(1024x768) which could show ubiquity's windows correctly. When we prepare to scale it with the dpi's rules, we judge if the window size after scale is bigger than the real resolution, so on 2560x1440, we don't scale, cause the 1440 is lower than 768x2, after scale it will out of screen.

https://bazaar.launchpad.net/~oem-solutions-group/kittyhawk/unity-settings-daemon_Seb.lp1382291/revision/18

[Mathieu Trudel-Lapierre (cyphermox)]

Patch looks fine to me, but so far there is nothing here for ubiquity. Do you need any changes to ubiquity after all?

[Bin Li (binli)]

@Mathieu,

 Sorry for reply late, looks I miss the last change.

 And the answer is No, the previous patch which change ubiquity was rejected by Seb, and Seb helped make the above patch(#72), my work just based on his patch. Thanks!

[Mathieu Trudel-Lapierre (cyphermox)]

Marking the ubiquity task as Invalid; let's switch it back if it turns out there are some needed changes in ubiquity (as for wallpaper in ubiquity-dm?)

[Shih-Yuan Lee (fourdollars)]

Ubiquity needs to enable the xsetings plugin of unity-settings-daemon or gnome-settings-daemon to set up the scale factor properly.
Or Ubiquity detects the HiDPI and handles it by itself.
No matter by which way they both need to modify Ubiquity anyway.

[Ara Pulido (ara)]

I have confirmed with the original reporters that this has been fixed in Xenial