[FFe] Sync (almost) with libreoffice-dictionaries in Debian sid

[Björn Michaelsen]

Proposed uploads are available in this PPA:
https://launchpad.net/~gunnarhj/+archive/ubuntu/lo-dicts2

This proposal gives us updated dictionaries in xenial, and quite a few new ones. libreoffice-dictionaries in Ubuntu was introduced before the release of trusty, but hasn't been maintained since. This proposal should make it possible to really sync with Debian after the xenial release.

[Original description]
libreoffice-dictionaries hasnt been synced from Debian for a while and has Ubuntu-local change. Rebase/Reimplement those on Debian and upstream those, if possible.

(libreoffice-dictionaries was created rather ad-hoc around 4.2 before Debian did, replacing the old openoffice-dictionaries, thus the missing updates since).

[Mattia Rizzolo (mapreri)]

are there ubuntu changes that make sense to keep at all?
I see the most relevant delta is the bunch of .dictlistinfo installed in
    $(CURDIR)/debian/$$pkg/usr/share/myspell/infos/ooo/$$pkg
that ttbomk are not used anymore nowadays.

as stated on #268953 I wanted to give a shoot at this now that xenial is open, but by all means I'll let you do it instead (at least the initial difficult merge).

[Björn Michaelsen (bjoern-michaelsen)]

I just looked if we can do a direct sync, but stopped when I found that Debian wasnt building hunspell-sv-se. I wouldnt merge if there is not more to it, rather just readd those on top of Debian package ..

[Mattia Rizzolo (mapreri)]

On Mon, Oct 26, 2015 at 07:52:11PM -0000, Björn Michaelsen wrote:
> I just looked if we can do a direct sync, but stopped when I found that
> Debian wasnt building hunspell-sv-se. I wouldnt merge if there is not
> more to it, rather just readd those on top of Debian package ..

debian's lo-dicts builds hunspell-sv, which conflicts with
hunspell-sv-se, which is built by another source package, also in
ubuntu.

Since I'm the debian maintainer of lo-dicts please just ask if you
notice something weird. Please also note that just syncing might not be
that wise. replaces/conflicts might be very well needed due to
myspell-${lang} → hunspell-${lang} transition or a lot of other reasons.
I'm up for adding those (versioned) conflicts directly in the debian
package if they don't clash with somthing)

--
regards,
                        Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`.
more about me: http://mapreri.org : :' :
Launchpad user: https://launchpad.net/~mapreri `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia `-

[Mattia Rizzolo (mapreri)]

are there any news on this?
would be nice to have this done for xenial, so we can drop conflicts and stuff right after it, instead of keeping them for 2 more years.

[Amr Ibrahim (amribrahim1987)]

Please, sync the latest with Debian and merge the changes.

[Mattia Rizzolo (mapreri)]

I'd love to see Björn Michaelsen do the initial merge (because I don't think a sync is possible). After that I'll happily take care of the subsequent uploads, but I don't have enough time to dedicate to the first, big one.

[Gunnar Hjalmarsson (gunnarhj)]

I don't think that hunspell-sv-se is a blocker for syncing from Debian.

Currently there are two versions of hunspell-sv-se in the Ubuntu archive: The latest is built by libreoffice-dictionaries, but there is also an old one built by the hunspell-sv source package. If we would sync, both would be overridden by hunspell-sv, and that's what we want, isn't it?

OTOH, libreoffice-dictionaries includes spellcheckers for both sv-se and sv-fi, so we should change it in debian so it builds both hunspell-sv-se and hunspell-sv-fi. But that can be done afterwards, if I understand it correctly. Since I'm from Sweden, I'll be happy to help sorting out any Swedish issues. ;)

It would be great if the sync with Debian could be done ASAP.

[Mattia Rizzolo (mapreri)]

Well, lo-dicts builds a binary hunspell-sv containing both dictionaries. the source hunspell-sv instead builds 2 binaries hunspell-sv-se and myspell-sv-se. They probably conflicts, but one doesn't take the other over automatically, no.

I'd be lovely see this done before xenial… Björn??

[Gunnar Hjalmarsson (gunnarhj)]

On 2016-02-10 16:34, Mattia Rizzolo wrote:
> Well, lo-dicts builds a binary hunspell-sv containing both dictionaries.

Aha, great, missed that. Then all is well, I think.

> the source hunspell-sv instead builds 2 binaries hunspell-sv-se and
> myspell-sv-se. They probably conflicts, but one doesn't take the other
> over automatically, no.

I decided that some testing is a good thing as a base for decision, so I created a PPA:

https://launchpad.net/~gunnarhj/+archive/ubuntu/lo-dicts

It includes libreoffice-dictionaries from Debian sid, with a few adjustments in debian/control.

I have identified two other source packages which need to be changed, so the PPA also includes modified builds of:

* openoffice.org-hyphenation
* language-selector

There are three kinds of debian/control adjustments in lo-dicts:

1. The version number of lo-dicts in Ubuntu is 1:4.2.1-0ubuntu3, so I
replaced the occurrences of "(<= 1:3.3.0-4)" in Breaks/Replaces with
"(<< 1:5)".

2. Added "Replaces" fields to new hunspell-* packages which conflict
with other packages. This gives them precedence over conflicting
packages, and at least in Ubuntu I think that feels safer,
considering that language support packages are typically pulled by
language-selector in Ubuntu, and not so often picked manually by the
users.

3. Added "Replaces"/"Conflicts" fields to new hyphen-* packages which
replace hyph_* dictionaries in openoffice.org-hyphenation 0.8.

After having enabled the PPA, I could smoothly upgrade with Synaptic and add new packages by opening gnome-language-selector.

@Mattia: Could you please review the changes I made to debian/control, and consider them for inclusion in the Debian repository.

With those changes, I think that a sync is not only possible, but also the preferred way to do it.

[Gunnar Hjalmarsson (gunnarhj)]

Hmm.. Time goes by, and lo-dicts 1:5.1.0-1 seems to be stuck in Debian's new queue for some reason. One possibility is that we simply upload in Ubuntu with a debian/control delta for now, to make reasonably updated dictionaries available in 16.04. However, it would still be good if we could agree on the debian/control changes, to prepare for sync going forward.

@Mattia: Would that be a reasonable plan?

@Björn: Do you have any doubts, given the tests I mentioned in comment #9?

I suppose this would require a feature freeze exception.

[Mattia Rizzolo (mapreri)]

So, I applied a version of the first point in git (with a different version that is not invented), that should be enough.

For the second one: I'm not sure I like adding that in debian at all. It shouldn't be necessary anyway (also considering that my evil plan includes pushing for the removals of those conflicting dictionaries in the long run), tbh.
In particular, the addition of the Conflicts to myspell-es is not necessary, the files don't conflicts

For the third: I'd rather upload a openoffice.org-hyphenation without the conflicting dictionaries once this one is uploaded and don't care more, it sounds cleaner, and maybe add a Breaks+Replaces on lo-dicts instead. The remaining hyphenation dicts in oo.o-hyphenation would be en_CA, es_ES (but this is really redundant), fi_FI, ga_IE, id_ID, is_IS; maybe I can get some of them in libreoffice-dictionaries in the future (way too late for xenial).

There is another thing that probably you missed: adding transitional packages from myspell-* to hunspell-*. I don't want to add them in debian for sure. This is maybe the only little thing that I veto against, so this is actually the only delta that should be needed, in debian/control.in, that can just go away once xenial is released.

[Gunnar Hjalmarsson (gunnarhj)]

Thanks for your comments, Mattia.

I have considered it further, and created a new PPA, this time with lo-dicts 1:5.1.0-1:

https://launchpad.net/~gunnarhj/+archive/ubuntu/lo-dicts4

This time I made use of the nifty debian/helper.py, and to start with I made the version change which you already made in git.

As you suggested I created a bunch of transitional packages. That required that the Conficts fields in the corresponding hunspell-* packages and the hyphen-no package were replaced with Breaks/Replaces. I agree it's not necessary to make those changes in Debian, since they can be dropped after xenial.

Please note that I made a transitional myspell-es package. The current myspell-es includes a Conflict field to hunspell-es, so the packages do conflict, even if there are no file conflicts.

I blacklisted hyphen-et for consistency reasons - hunspell-et is blacklisted - and deleted debian/hyphen-et.install. You may want to consider these changes for Debian. Otherwise, if someone installs hyphen-et, myspell-et gets deleted, and they have no access to a spellchecker.

As regards openoffice.org-hyphenation, I think you are right: The Conflicts/Replaces fields I first suggested would be unnecessary.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Mattia Rizzolo (mapreri)]

On Tue, Mar 01, 2016 at 09:23:08PM -0000, Gunnar Hjalmarsson wrote:
> As you suggested I created a bunch of transitional packages. That
> required that the Conficts fields in the corresponding hunspell-*
> packages and the hyphen-no package were replaced with Breaks/Replaces. I
> agree it's not necessary to make those changes in Debian, since they can
> be dropped after xenial.
>
> Please note that I made a transitional myspell-es package. The current
> myspell-es includes a Conflict field to hunspell-es, so the packages do
> conflict, even if there are no file conflicts.

umh, actually I was talking about the migrating away from the myspell-*
packages built by lo-dicts, I didn't check the list of those produced by
ubuntu, but now I see there is a missing transitional package for
myspell-en-us. I recall now that I wanted to move that to
src:hunspell-en-us, but totally forgot to do.

If we do that we would take over several binaries from other sources,
that means that some of those sources may become pointless and just be
removed from the archive (in ubuntu). That needs to be backed by
somebody else, I can't really help here (it was a way longer term plan
for me), and not sure we would do that in during FF.
It would indeed be nice to kick all those other packages, though.

> I blacklisted hyphen-et for consistency reasons - hunspell-et is
> blacklisted - and deleted debian/hyphen-et.install. You may want to
> consider these changes for Debian. Otherwise, if someone installs
> hyphen-et, myspell-et gets deleted, and they have no access to a
> spellchecker.

yeah, nice catch, incorporated :)

[Gunnar Hjalmarsson (gunnarhj)]

On 2016-03-02 11:32, Mattia Rizzolo wrote:
> umh, actually I was talking about the migrating away from the
> myspell-* packages built by lo-dicts, I didn't check the list of
> those produced by ubuntu, but now I see there is a missing
> transitional package for myspell-en-us. I recall now that I wanted to
> move that to src:hunspell-en-us, but totally forgot to do.

I see.

In Ubuntu it's most important that hunspell-en-us is removed from the blacklist and built, isn't it? A transitional myspell-en-us package shouldn't be necessary in Ubuntu any longer (but wouldn't hurt).

> If we do that we would take over several binaries from other
> sources, that means that some of those sources may become pointless
> and just be removed from the archive (in ubuntu).

That's true. Some of the myspell-* packages, which would be overridden this way, are quite old and poorly maintained. Assuming that the dictionaries provided by lo-dicts are more up-to-date, it would be a convenient way to get rid of them. Admittedly a bold way.

> That needs to be backed by somebody else, I can't really help here
> (it was a way longer term plan for me), and not sure we would do that
> in during FF.
> It would indeed be nice to kick all those other packages, though.

Letting hunspell-* packages co-exist in the archive with conflicting myspell-* packages, as you now do in Debian, doesn't fit very well with how language-selector pulls those packages. Without the transitional packages, the users would be prompted to install hunspell-* packages anyway. Of course, they have the option to not install them.

OTOH, maybe we could see this as a 'soft transition' away from old myspell-* packages. Ubuntu users would be prompted to install hunspell-*, and most would just accept. If there proves to be many users who prefer a conflicting myspell-* package, for whatever reason, some of them would find the language-selector prompting irritating, and let us know.

Is that a reasonable way to look at it?

[Martin Pitt (pitti)]

From a release team's POV, I still consider this appropriate cleanup for xenial at this time (i. e. FF approved if you are after that). Indeed *not* doing this has regressed language support over the time, so reducing this delta and getting back in sync with Debian and the more modern hunspell dicts seems entirely desirable, and the risk seems manageable.

[Gunnar Hjalmarsson (gunnarhj)]

Setting status "Triaged" due to Martin's FFe approval.

[Gunnar Hjalmarsson (gunnarhj)]

Mattia,

Well, I got it all wrong about myspell-en-us. Sorry.

The latest (last?) attempt at an uploadable set of packages are available here:

https://launchpad.net/~gunnarhj/+archive/ubuntu/lo-dicts2

Basically the Ubuntu/Debian delta in lo-dicts consists of two transitional packages: myspell-en-us and hunspell-sv-se. The myspell-en-us transitional package made it necessary to adjust debian/control in hunspell-en-us.

I think that's it. Really.

Would appreciate your comments before I ask for upload sponsoring.

[Mattia Rizzolo (mapreri)]

To ease this, I just uploaded to debian a -2 including some of those
changes, so maybe rebase.

> Basically the Ubuntu/Debian delta in lo-dicts consists of two
[...]
> Would appreciate your comments before I ask for upload sponsoring.

I think it makes a lot of sense, please try going ahead. As Martin
said, the risks are low, and anything that would come up in the next
weeks should be fairly easy to fix (and then, both me and you are
subscribed to the bugs of this package)...

[Gunnar Hjalmarsson (gunnarhj)]

Nah, the important stuff in that new Debian upload is already included in the lo-dicts source in the PPA, so rebasing wouldn't be worth the extra work. We'll still be able to truly sync after the Xenial release, so let's stick with what's there now.

I just uploaded language-selector, and asked for assistance with the other packages.

Thanks for your kind cooperation in this matter, Mattia!

[Launchpad Janitor (janitor)]

This bug was fixed in the package language-selector - 0.158

---------------
language-selector (0.158) xenial; urgency=medium

  * data/pkg_depends:
    - Stop pulling myspell-* for languages where hunspell-* are made
      available via libreoffice-dictionaries 1:5.1.0-1ubuntu1
      (LP: #1510198).
    - Stop pulling openoffice.org-hyphenation for languages which are
      not available in version 0.9.
    - Pull hunspell-no/hyphen-no/mythes-no if nb_NO or nn_NO locale.
  * language_support_pkgs.py:
    - Hack to prevent that hunspell-sv-se is pulled by
      check-language-support.

 -- Gunnar Hjalmarsson <email address hidden> Fri, 04 Mar 2016 02:57:00 +0100

[Björn Michaelsen (bjoern-michaelsen)]

Sorry, was blocked on other tasks. Reading the discussion here, going ahead sounds fine for me. Anything still needed from my side?

[Amr Ibrahim (amribrahim1987)]

Great job everyone. I would love to see this in xenial :)

[Gunnar Hjalmarsson (gunnarhj)]

On 2016-03-04 17:24, Björn Michaelsen wrote:
> Reading the discussion here, going ahead sounds fine for me.

Excellent, Björn.

> Anything still needed from my side?

Well, you could nudge a core-dev to take it on soon, so we'll have sufficient time before final freeze to handle possible issues. Besides that, I think we have it under control.

[Mattia Rizzolo (mapreri)]

\o/

Thanks you all!

[Launchpad Janitor (janitor)]

This bug was fixed in the package hunspell-en-us - 20070829-6ubuntu3

---------------
hunspell-en-us (20070829-6ubuntu3) xenial; urgency=medium

  * debian/control:
    Add <= version to myspell-en-us Replaces/Conflicts (LP: #1510198).

 -- Gunnar Hjalmarsson <email address hidden> Thu, 03 Mar 2016 23:28:00 +0100

[Launchpad Janitor (janitor)]

This bug was fixed in the package openoffice.org-hyphenation - 0.9

---------------
openoffice.org-hyphenation (0.9) xenial; urgency=low

  * Removed since they are now in hyphen-* packages built by
    libreoffice-dictionaries 1:5.1.0-1ubuntu1: cs_CZ, da_DK, el_GR,
    en_GB, es_ES, is_IS, nl_NL, pt_BR, pt_PT, sk_SK, uk_UA
    (LP: #1510198).

 -- Gunnar Hjalmarsson <email address hidden> Thu, 03 Mar 2016 22:42:00 +0100

[Launchpad Janitor (janitor)]

This bug was fixed in the package libreoffice-dictionaries - 1:5.1.0-1ubuntu1

---------------
libreoffice-dictionaries (1:5.1.0-1ubuntu1) xenial; urgency=medium

  * debian/helper.py:
    - Changes already in Debian git (LP: #1510198):
      - Bump version in Breaks/Replaces fields.
      - Blacklist hyphen-et, since hunspell-et is blacklisted.
    - Replace Conflicts fields with Breaks/Replaces in hunspell-sv.
  * debian/hyphen-et.install
    - Removed.
  * debian/control.in:
    - Additional transitional dummy packages (LP: #1510198):
      - myspell-en-us
      - hunspell-sv-se
    - Changed "Maintainer" to Ubuntu Developers.

 -- Gunnar Hjalmarsson <email address hidden> Thu, 03 Mar 2016 22:34:00 +0100