[FFe]Please provide 'locales-all' as in Debian

Bug #1394929 reported by Sylvain Beucler on 2014-11-21
20
This bug affects 3 people
Affects Status Importance Assigned to Milestone
glibc (Ubuntu)
Undecided
Unassigned
langpack-locales (Ubuntu)
Undecided
Unassigned

Bug Description

Repurposing this bug as a Feature Freeze exception request to merge langpack-locales into glibc, and make locales-all available in the process. Working on getting that all done and properly tested by tonight, but if this doesn't make it, it'll happen in early 16.10 instead.

[Original Report]
Hi,

Debian provides a 'locales-all' package, which is very convenient to ensure all locales are available, and installs very quickly.

It's particularly handy when providing i18n'd web applications (e.g. that use php-gettext).
As they may be accessed by visitors of all supported languages, rather than documenting how the server administrator should enable a list of 10-15 languages by reconfiguring 'locales', the packager can just add 'locales-all' to the Recommends or Depends.
See for instance the FusionForge package in Debian (https://packages.debian.org/jessie/gforge-web-apache2).

Cheers!
Sylvain

Thorsten Glaser (mirabilos) wrote :

I also require this package, see: http://askubuntu.com/q/435954/222799

Changed in glibc (Ubuntu):
status: New → Confirmed
Matthias Klose (doko) wrote :

the package is not required. you can generate the locales on the fly for package builds (or build-depending on various language-pack packages).

$ cat locale-gen
#!/bin/sh

LOCPATH=`pwd`/locales
export LOCPATH

[ -d $LOCPATH ] || mkdir -p $LOCPATH

umask 022

echo "Generating locales..."
while read locale charset; do
        case $locale in \#*) continue;; esac
        [ -n "$locale" -a -n "$charset" ] || continue
        echo -n " `echo $locale | sed 's/\([^.\@]*\).*/\1/'`"
        echo -n ".$charset"
        echo -n `echo $locale | sed 's/\([^\@]*\)\(\@.*\)*/\2/'`
        echo -n '...'
        if [ -f $LOCPATH/$locale ]; then
            input=$locale
        else
            input=`echo $locale | sed 's/\([^.]*\)[^@]*\(.*\)/\1\2/'`
        fi
        localedef -i $input -c -f $charset $LOCPATH/$locale #-A /etc/locale.alias
        echo ' done'; \
done <<EOF
de_DE ISO-8859-1
de_DE@euro ISO-8859-15
en_HK ISO-8859-1
en_US.UTF-8 UTF-8
es_ES ISO-8859-1
EOF

echo "Generation complete."

$ LOCPATH=... <run your command>

Sylvain Beucler (beuc) wrote :

Hi,

I agree that there's way to work-around this missing package, but:
- this is much slower, and
- the fact that "locales-all" is not available on Ubuntu requires the project to package a number of packages from Debian.
- also I wouldn't feel safe automating this work-around, for instance I've got cases where the admin just didn't want to install a particular locale due to incorrect translations. Having "locales-all" in Recommends: allowed the necessary flexibility.

Is there a particular reason why Ubuntu shouldn't ship this package?

Sylvain Beucler (beuc) wrote :

"requires the project to package a number of packages from Debian."
I meant requires to patch.

Thorsten Glaser (mirabilos) wrote :

Hi Doko,

the package *is* required if you programmatically need to have “all”, or at least a large set of, locales available.

Possible use case here is a package dependency (declaratively).

In the specific case of FusionForge, I have seen it (both the native tracker and the embedded mediawiki) malfunction if locales-all was not installed, even getting PHP to segfault.

Another use case is to ensure something is available to users, especially nōn-root users, in the default location (so there is no need to set LOCPATH. (This can help ssh, sudo, etc.)

Sylvain Beucler (beuc) wrote :

Hi,

Is there any progress on this?
You didn't answer me.. :/

Gunnar Hjalmarsson (gunnarhj) wrote :

Martin Pitt just let me know on IRC that there is a plan to move locales in Ubuntu from the langpack-locales source package to the glibc source package. In connection with that, langpack-all would be made available in Ubuntu.

Gunnar Hjalmarsson (gunnarhj) wrote :

s/langpack-all/locales-all/

Hans Joachim Desserud (hjd) wrote :

>there is a plan to move locales in Ubuntu from the langpack-locales source package to the glibc source package. In connection with that, langpack-all would be made available in Ubuntu.

Hi, do you have a rough time estimate on this transition?

I see from http://qa.ubuntuwire.com/ftbfs that at least groovy2 requires locales-all in order to build. I don't know whether it is strictly needed or if something like the workaround from #2 could be used instead.

Gunnar Hjalmarsson (gunnarhj) wrote :

On 2014-12-27 17:56, Hans Joachim Desserud wrote:
> On 2014-12-04 16:10, Gunnar Hjalmarsson wrote:
>> there is a plan to move locales in Ubuntu from the langpack-locales
>> source package to the glibc source package. In connection with
>> that, locales-all would be made available in Ubuntu.
>
> Hi, do you have a rough time estimate on this transition?

No. My personal guess is that it's not a high priority thing.

Hans Joachim Desserud (hjd) wrote :

>> Hi, do you have a rough time estimate on this transition?

>No. My personal guess is that it's not a high priority thing.

I guessed the same, but thought I'd check. Thanks. :)

Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in langpack-locales (Ubuntu):
status: New → Confirmed
Andrey Galkin (andvgal) wrote :

Also run into this in scope of automatic unified provisioning of Ubuntu and Debian systems.
Is it really a difficult task to import already existing Debian package?

Adam Conrad (adconrad) on 2016-03-18
summary: - Please provide 'locales-all' as in Debian
+ [FFe]Please provide 'locales-all' as in Debian
description: updated
Adam Conrad (adconrad) wrote :

Here's the diff between the 2.23 update (bug #1521172) and the locales merge proposed here.

Steve Langasek (vorlon) wrote :

@@ -96,7 +96,8 @@
 Section: localization
 Priority: standard
 Depends: libc-bin (>> 2.23), ${misc:Depends}, debconf | debconf-2.0
-Replaces: manpages-fr-extra (<< 20141022)
+Breaks: libc-bin (<< 2.23)
+Replaces: libc-bin (<< 2.23), manpages-fr-extra (<< 20141022)
 Build-Profiles: <!stage1>
 Description: GNU C Library: National Language (locale) data [support]
  Machine-readable data files, shared objects and programs used by the

This introduces a Breaks/Replaces between two packages in the minimal set (locales, libc-bin). This will warrant upgrade testing from both 14.04 and 15.10. Has any upgrade testing been done already?

The Breaks/Replaces from locales to libc-bin is definitely required, due to the move of the update-locale script.

I see the breaks relationship is also bidirectional, with libc6 Breaks: (locales << 2.23), which further emphasizes the need for upgrade testing. Why is the breaks in that direction required? Is this related to the drop of the locale-gen --purge call from libc.postinst?

Otherwise, everything in this looks good to me. It appears to be a faithful translation of the existing langpack handling code in the Ubuntu-specific package, and where there is delta it makes sense in terms of bugfixing of the existing behavior (e.g., properly splitting the handling of /etc/locale.gen vs. /var/lib/locales/supported.d, and fixing handling of langpacks to not modify contents of /usr/lib/locale, so that the set of available locales is always the union of the debconf-configured locales and the langpacks).

Adam Conrad (adconrad) wrote :

14:28 <infinity> Breaks loops are pretty friendly (compared to Conflicts loops), they just both deconfigure, both upgrade, and both configure.
14:28 <infinity> I tested from 2.21, and it went fine, I can't see why it would be worse from older versions.
14:29 <infinity> (The libc->Breaks locales relationship is standard in the Debian packaging to make sure the tools match libc, I was just adding it back)

tags: added: block-proposed
Adam Conrad (adconrad) wrote :

Adding the block-proposed tag until more upgrade testing from trusty has been done.

Steve Langasek (vorlon) wrote :

Discussed with Adam that upgrade testing can happen after the package has been uploaded to xenial-proposed, so that this doesn't need to block the official builds, and can happen in parallel with the autopkgtest testing.

So this just needs a blocking bug opened, to be closed only once the upgrade testing from both wily and trusty has been done.

Adam Conrad (adconrad) wrote :

Upgrade testing has been done, and looks good. Dropping the block-proposed tag.

tags: removed: block-proposed
Launchpad Janitor (janitor) wrote :
Download full text (41.1 KiB)

This bug was fixed in the package glibc - 2.23-0ubuntu1

---------------
glibc (2.23-0ubuntu1) xenial; urgency=medium

  * Merge with 2.23 from experimental (LP: #1521172), remaining changes:
    - Remove the libc6 recommends on libc6-i686, which we don't build.
    - Enable libc6{,-dev}-armel on armhf and libc6{-dev}-armhf on armel.
    - Ship update-locale and validlocale in /usr/sbin in libc-bin.
    - Don't build locales or locales-all in Ubuntu, we rely on langpacks.
    - Heavily mangle the way we do service restarting on major upgrades.
    - Build i386 variants as -march=i686, build amd64 with -O3, and build
      ppc64 variants (both 64-bit and 32-bit) with -O3 -fno-tree-vectorize.
    - Build generic i386/i686 flavour with -mno-tls-direct-seg-refs.
    - Drop the libc6-xen flavour, as the above change covers Xen's needs.
    - Enable systemtap support, which is currently disabled in Debian.
    - debian/patches/ubuntu/local-warn-bsd-source.diff: Don't warn on
      _BSD_SOURCE || _SVID_SOURCE, so we don't have to fix everything
      that might define them and build with -Werror, drop after xenial.
    - Don't build libc-l10n, its contents get stripped for language-packs.
    - debian/control.in/libc: Remove a Breaks on s390x against a binary
      NMU version of check that will never exist in the Ubuntu archive.
  * Drop libc-bin manpages Recommends to Suggests to keep it in standard.
  * Build with -fno-pie and -no-pie on s390x, upstream isn't PIE-ready.
  * debian/testsuite-xfail-debian.mk: XFAIL tst-malloc-thread-{fail,exit}.
  * debian/patches/any/cvs-netinet-in-4.2.diff: Dropped, merged upstream.
  * debian/patches/any/cvs-netinet-tcp-4.2.diff: Dropped, merged upstream.
  * debian/patches/any/cvs-selinux-nscd.diff: Dropped, merged upstream.
  * debian/patches/powerpc/local-tune-power8.diff: Dropped, we build for P8.
  * debian/patches/ubuntu/submitted-no-stack-backtrace.diff: Rebase patch.
  * debian/patches/ubuntu/unsubmitted-increase-dtv-surplus.diff: Dropped.

  * Evaluate and merge/drop/rebase localedata patches from langpack-locales.
    - debian-*patch: Drop all langpack-locales patches copied from Debian.
    - ubuntu/localedata/bn-language.patch: Rebase for glibc.
    - ubuntu/localedata/ce_RU-new_locale.patch: Drop, merged upstream.
    - ubuntu/localedata/ckb_IQ-new_locale.patch: Rebase for glibc.
    - ubuntu/localedata/de_AT-february.patch: Drop, merged upstream.
    - ubuntu/localedata/en_CA-dateformat.patch: Rebase for glibc.
    - ubuntu/localedata/en_IE-am_pm.patch: Rebase for glibc.
    - ubuntu/localedata/eo_US.patch: Rebase for glibc.
    - ubuntu/localedata/es-decimal_point-thousands_sep.patch: Drop, merged.
    - ubuntu/localedata/es_DO-am_pm.patch: Rebase for glibc.
    - ubuntu/localedata/es_EV.patch: Drop, merged upstream.
    - ubuntu/localedata/gd_GB-days-months.patch: Drop, merged upstream.
    - ubuntu/localedata/it_IT-thousands_sep.patch: Drop, also in Debian.
    - ubuntu/localedata/ln_CD-new_locale.patch: Rebase for glibc.
    - ubuntu/localedata/lt_LT-currency.patch: Drop, merged upstream.
    - ubuntu/localedata/lv_LV-currency.patch: Rebase for glibc.
    - ubuntu/localedata/pt_BR-d_fmt.patch: ...

Changed in glibc (Ubuntu):
status: Confirmed → Fix Released
Adam Conrad (adconrad) on 2016-03-24
Changed in langpack-locales (Ubuntu):
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers