[META] Handling Japanese new era "令和 (Reiwa)"

[Mathieu Trudel-Lapierre]

[Background]
Many packages are affected by the requirement to support the new era "Reiwa" (令和）

This is the meta bug to track packages that need fixes; which packages have already been SRUd to previous releases, how to prioritize the work needed, and general test cases for verifying that things are working as expected.

[Impact]
Users who run Ubuntu in Japanese.

[Test cases]

== Date conversion ==

On applications that support writing dates in long form, or with symbols to denote era (either in X00.00.00 format or in GG1G5G1G format (G- glyph; X- character):

1) Enable date formatting in each of the above formats that are supported (long form or symbols)
2) Type in '2019/05/01' to be formatted, verify that it shows as "令和1年5月1日" or "R1.05.01"
3) Type in '2019/04/30' to be formatted, verify that it shows as "平成31年4月30日" or "H31.4.30"

== Date output ==

1) Set date to 2019/05/01
2) Output date; verify that the year it is displayed as "令和元年"
3) Set date to 2019/04/30
4) Output date; verify that the year is diplayed as "平成31年"

=== Displaying formatted year for Japanese era with glibc ===

Run:
LC_ALL=ja_JP.utf8 date +%EY -d 20190430 # previous era (should still work as before SRUs)
or
LC_ALL=ja_JP.utf8 date +%EY -d 20190501 # new era (should now correctly display the new era)

== Character maps / font support ==

1) Search for character "SQUARE ERA NAME"
2) Verify that the results include at least "SQUARE ERA NAME HEISEI" and "SQUARE ERA NAME REIWA" (there should also be Syouwa, Taisyou and Meizi), and that the glyphs are readable:

 - SQUARE ERA NAME HEISEI: ㍻
 - SQUARE ERA NAME REIWA: 令和 (in a single glyph)

Display of the Reiwa square glyph is font-specific; it may show simply as a empty square or a square with hex characters. If that is the case, the unicode data supports the new character, but the selected font does not include the new glyph.

== Typing / input methods ==

1) Type in 'heisei'
2) Verify that the input method in use gives you an option "平成", and optionally also the square era glyph.
3) Type in 'reiwa'
4) Verify that the input method in use gives you an option that includes "令和", and possibly also the square era glyph (if supported for Heisei)

5) Type in the following strings, and verify that the options are provided in the input method:
    * "れいわ"(reiwa) => "令和"
    * "れいわ"(reiwa) => "㋿"
    * "２０１８ねん"(2018nenn) => "平成三十年"
    * "２０１９ねん"(2019nenn) => "平成三十一年"
    * "２０１９ねん"(2019nenn) => "令和元年"
    * "２０２０ねん"(2020nenn) => "令和二年"

/!\ Some fonts, like fonts-noto-cjk, currently have no glyph for U+32FF.

[Regression potential]
This is a potentially large change as it impacts font display, character sets as well as date conversions. As such, extreme care should be taken to ensure that regressions are avoided, such that dates previous to May 1, 2019 continue to display as before, and dates onward are displayed with the new era symbols. The included test cases account for verifying the continued behavior or previous dates.

[Mathieu Trudel-Lapierre (cyphermox)]

mozc appears to already be Fix Released for most releases; this has been handled as bug 1823444.

[Mathieu Trudel-Lapierre (cyphermox)]

libreoffice SRUs appear to be handled in bug 1827451.

[Gunnar Hjalmarsson (gunnarhj)]

Added a task for fonts-noto-cjk, which is the default font package in Ubuntu for Japanese since 18.04; please see this list message:

https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2019-May/018311.html

No news on the Debian side yet.

[Mitsuya Shibata (cosmos-door)]

"LC_ALL=ja_JP.utf8 date +%EY -d 20190501" depends on /usr/share/i18n/locales/ja_JP in locales package (src:glibc).

And other related packages are listed here.

  https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2019-May/018309.html

[Mathieu Trudel-Lapierre (cyphermox)]

Oops; picked the wrong openjdk...

FWIW; according to the email by Mitsuya Shibata, openjdk 8 and 11 at least are affected (so, everything prior to disco if updates are not applied. Openjdk-8 updates appear to already be at 8u212, which should include Reiwa support. Marking as Fix Released so we can see at a glance that it's already been covered.

[Mathieu Trudel-Lapierre (cyphermox)]

unicode-data in eoan does include Reiwa. Reverse-depends probably still need to be rebuilt (I'm testing gucharmap which seemed easy enough to patch to work).

[Mathieu Trudel-Lapierre (cyphermox)]

mozc appears to be all done (LP: #1823444)

[Gunnar Hjalmarsson (gunnarhj)]

Dropping the fonts-noto-cjk Xenial task, since Noto is not default font for Japanese in Xenial. (Takao is.)

[Mathieu Trudel-Lapierre (cyphermox)]

Looks like noto sources now have the right glyphs (since their April 9 release, actually). It will need an update both in Debian and Ubuntu.

[Gunnar Hjalmarsson (gunnarhj)]

On 2019-06-21 15:59, Mathieu Trudel-Lapierre wrote:
> Looks like noto sources now have the right glyphs (since their April
> 9 release, actually).

Right. There are 2 months old merge requests in Debian to get them in:

https://salsa.debian.org/fonts-team/fonts-noto-cjk/merge_requests

But nobody has reacted to those requests yet AFAIK.

[Gunnar Hjalmarsson (gunnarhj)]

fonts-noto-cjk fixed in disco and bionic via bug #1834406.

[Mathieu Trudel-Lapierre (cyphermox)]

gucharmap split up into bug 1838321

[Mathieu Trudel-Lapierre (cyphermox)]

icu split up into bug 1838322.

[Mathieu Trudel-Lapierre (cyphermox)]

Split up unicode-data into its own bug; bug 1838323.

[Will Cooke (willcooke)]

Marking no longer affecting poppler-data. If there is still a problem it should be logged as a new bug.