axi-cache conversion to python3 broke the script

[Tannoiser]

axi-cache, the script command line in apt-xapian-index, is now converted to python3.
In python3 afaik module ConfigParser is now renamed in configparser.

running axi-cache, throw an error:

mizio@sammiz:~$ axi-cache
No module named 'ConfigParser': only help functions are implemented, for the sake of help2man

if I edit axi-cache, fixing the module name (row 50, "from configparser import RawConfigParser" instead of "from ConfigParser..." all seems ok:

mizio@sammiz:~$ axi-cache
Usage: axi-cache [options] command [args]

Query the Apt Xapian index.

Commands:
<...>

But this fix isn't sufficient. The script not seems to me really python3 compatible, ie:

mizio@sammiz:~$ axi-cache search brasero
4 results found.
Results 1-4:
100% b'brasero-cdrkit' - (unknown by apt)
98% b'brasero-common' - (unknown by apt)
77% b'gir1.2-brasero-3.0' - (unknown by apt)
71% b'brasero' - (unknown by apt)
Traceback (most recent call last):
  File "/usr/bin/axi-cache", line 852, in <module>
    sys.exit(ui.perform())
  File "/usr/bin/axi-cache", line 847, in perform
    return f(self.args)
  File "/usr/bin/axi-cache", line 547, in do_search
    self.print_matches(self.db.get_matches())
  File "/usr/bin/axi-cache", line 808, in print_matches
    sugg = self.clean_suggestions(self.db.get_suggestions(count=7, filter=DB.TermFilter(stemmer=self.db.stem, exclude=self.args)))
  File "/usr/bin/axi-cache", line 310, in get_suggestions
    return self.enquire.get_eset(count, rset, filter)
  File "/usr/bin/axi-cache", line 118, in __call__
    return term[0].islower()
AttributeError: 'int' object has no attribute 'islower'

for this reason I suggest to revert the "python3 conversion" or ask the upstream to convert the script.

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: apt-xapian-index 0.47ubuntu8.1 [modified: usr/bin/axi-cache]
ProcVersionSignature: Ubuntu 4.4.0-35.54-generic 4.4.16
Uname: Linux 4.4.0-35-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.1
Architecture: amd64
CurrentDesktop: GNOME
Date: Sat Aug 13 14:39:38 2016
InstallationDate: Installed on 2010-05-06 (2290 days ago)
InstallationMedia: Ubuntu 10.04 LTS "Lucid Lynx" - Release amd64 (20100429)
PackageArchitecture: all
SourcePackage: apt-xapian-index
UpgradeStatus: Upgraded to xenial on 2016-08-12 (0 days ago)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Norbert (nrbrtx)]

Got this bug on Xenial amd64 with all updates.
I have apt-xapian-index 0.47ubuntu8.3 installed.

I reinstalled python-configparser.
After editing of line 50:
change "from ConfigParser import RawConfigParser" to "from configparser import RawConfigParser" I get.

`axi-cache info` works without error,

`axi-cache search brasero` produce error:

$ axi-cache search brasero
4 results found.
Results 1-4:
100% b'brasero-common' - (unknown by apt)
98% b'brasero-cdrkit' - (unknown by apt)
74% b'gir1.2-brasero-3.0' - (unknown by apt)
69% b'brasero' - (unknown by apt)
Traceback (most recent call last):
  File "/usr/bin/axi-cache", line 852, in <module>
    sys.exit(ui.perform())
  File "/usr/bin/axi-cache", line 847, in perform
    return f(self.args)
  File "/usr/bin/axi-cache", line 547, in do_search
    self.print_matches(self.db.get_matches())
  File "/usr/bin/axi-cache", line 808, in print_matches
    sugg = self.clean_suggestions(self.db.get_suggestions(count=7, filter=DB.TermFilter(stemmer=self.db.stem, exclude=self.args)))
  File "/usr/bin/axi-cache", line 310, in get_suggestions
    return self.enquire.get_eset(count, rset, filter)
  File "/usr/bin/axi-cache", line 118, in __call__
    return term[0].islower()
AttributeError: 'int' object has no attribute 'islower'

`axi-cache rdetails meld` produce error:
$ axi-cache rdetails meld
Traceback (most recent call last):
  File "/usr/bin/axi-cache", line 852, in <module>
    sys.exit(ui.perform())
  File "/usr/bin/axi-cache", line 847, in perform
    return f(self.args)
  File "/usr/bin/axi-cache", line 714, in do_rdetails
    print(name, tag, " ".join(deps))
TypeError: sequence item 0: expected str instance, bytes found

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "this patch is not sufficient, but needed" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Norbert (nrbrtx)]

My patch may be applied as
sudo sed -i 's/from ConfigParser import RawConfigParser/from configparser import RawConfigParser/' /usr/bin/axi-cache
but this is not enough.

What is wrong with version from Debian Stretch (https://packages.debian.org/stretch/apt-xapian-index)? Why you migrated to python3? I can't understand.

Manually installed version from Debian do not crash and works as expected.
(
$ axi-cache search brasero
4 results found.
Results 1-4:
100% brasero-common - Common files for the Brasero CD burning application and library
98% brasero-cdrkit - cdrkit extensions for the Brasero burning application
74% gir1.2-brasero-3.0 - CD/DVD burning library for GNOME - GObject introspection data
69% brasero - CD/DVD burning application for GNOME
More terms: burning application cdrkit gnome introspection gir1.2 extensions
More tags:

$ axi-cache again
4 results found.
Results 1-4:
100% brasero-common - Common files for the Brasero CD burning application and library
98% brasero-cdrkit - cdrkit extensions for the Brasero burning application
74% gir1.2-brasero-3.0 - CD/DVD burning library for GNOME - GObject introspection data
69% brasero - CD/DVD burning application for GNOME
More terms: brasero burning application cdrkit gnome introspection gir1.2
More tags:

$ axi-cache last
4 results found.
Results 1-4:
100% brasero-common - Common files for the Brasero CD burning application and library
98% brasero-cdrkit - cdrkit extensions for the Brasero burning application
74% gir1.2-brasero-3.0 - CD/DVD burning library for GNOME - GObject introspection data
69% brasero - CD/DVD burning application for GNOME
More terms: brasero burning application cdrkit gnome introspection gir1.2
More tags:

$ axi-cache more
4 results found.
No more results to show
`axi-cache again' will restart the search

$ axi-cache rdetails brasero
brasero dep gnome ubuntu-gnome-desktop ubuntu-mate-desktop
brasero rec ubuntustudio-video aptoncd banshee cinnamon-desktop-environment
brasero sug furiusisomount exfalso sound-juicer uck nautilus fotoxx
brasero bre brasero-common
)

[Khurshid Alam (khurshid-alam)]

Yes they have broken it. They didn't convert byte to string properly. I have made a patch...or rather hack really.

After applying the patch try search with colon:

axi-cache search 'brasero'

But it won't show package details for that, it needs to be fixed properly. You could try contacting them.

[Norbert (nrbrtx)]

What is interesting - during reinstallation of apt-xapian-index 0.47 from xenial here is interesting message:
"The index /var/lib/apt-xapian-index/index is not CJK-compatible, rebuilding it"

I installed version 0.49 from Debian Stretch. It does not report any specific messages:
$ sudo dpkg -i apt-xapian-index_0.49_all.deb
Selecting previously unselected package apt-xapian-index.
(Reading database ... 662973 files and directories currently installed.)
Preparing to unpack apt-xapian-index_0.49_all.deb ...
Unpacking apt-xapian-index (0.49) ...
Setting up apt-xapian-index (0.49) ...
apt-xapian-index: Building new index in background... # <- it is expected
Processing triggers for man-db (2.7.5-1) ...

I'll test it and report back. I installed `muon` for comparison with Synaptic.

[Norbert (nrbrtx)]

Artful and Zesty are affected too.
axi-cache info does not work, with same error message - "No module named 'ConfigParser': only help functions are implemented, for the sake of help2man".

[Norbert (nrbrtx)]

In Artful bug is not fixed.

[Norbert (nrbrtx)]

Ubuntu 17.10 with all updates - bug is not fixed.

[Norbert (nrbrtx)]

Ubuntu 17.10 final with all updates - bug is not fixed.

[Norbert (nrbrtx)]

Bug exist in bionic with all todays updates.

[dino99 (9d9)]

Is that change could be related ?

apt-xapian-index (0.47ubuntu8) xenial; urgency=medium

  * debian/patches/05_python3.patch: del the indexer instance explicitly
    before sys.exit(), so that it gets decrefed and freed before Python's
    implicit shutdown machinery can potentially leave an empty os module for
    ServerProgress.__del__() to find. (LP: #1530518)

 -- Barry Warsaw <email address hidden> Wed, 13 Apr 2016 17:36:22 -0400

[Norbert (nrbrtx)]

I am not to familiar with apt-xapian-index.
I have apt-xapian-index 0.47ubuntu8.3 from xenial-updates/universe repository.

I know only that current Xenial's version of `axi-cache` is broken:

$ axi-cache
No module named 'ConfigParser': only help functions are implemented, for the sake of help2man

I do not know how it communicates with Synaptic (or Muon), but this communication seems to be broken.

[Carlo Vanini (silhusk)]

The package from Debian works (as mentioned in comment #5) because it is the "original" Python2.7 version. The package in Ubuntu includes a (incomplete) port to Python3. Now, going back to Python2 would probably "fix" these problems, but Python2 is being phased out in the upcoming Ubuntu releases. Therefore, moving to Python3 does actually make sense, but there is no effort upstream being put into axi. It should probably disappear.

I suggest you use `apt search` and `apt show` instead of `axi-cache`.

Is there anything that `axi-cache` can do and cannot be done with `apt`?

[Norbert (nrbrtx)]

Dear Carlo!
> "Is there anything that `axi-cache` can do and cannot be done with `apt`?"
Are you writing this seriously?
`axi-cache` is broken and it does not matter that `apt` (or `apt-cache search`) is not.

All `apt-xapian-index` package is broken, it causes Synaptic (see bug 1685376) and Muon (see bug 1741767) to rebuild search indexes often and non-efficiently.

[Carlo Vanini (silhusk)]

Hello Norbert,

this bug report is about `axi-cache` not being fully ported. `axi-cache` is a front-end for querying the xapian index and finding packages, therefore I was suggesting a working alternative for the same task and asking whether any feature was missing.

[Norbert (nrbrtx)]

18.04 LTS with all updates, bug exists.

[dino99 (9d9)]

On bionic, getting rid of that error (dirty tweak until proper fix):

oem@ubuntu:~$ axi-cache
" No module named 'ConfigParser': only help functions are implemented, for the sake of help2man "

***********
First find location:
- ConfigParser.py from /usr/lib/python2.7
- configparser.py from /usr/lib/python3.6

As the axi-cache script request ConfigParser.py, then give it satisfaction:
" sudo cp /usr/lib/python3.6/configparser.py /usr/lib/python3.6/ConfigParser.py "

Then tested with "axi-cache info" & "axi-cache show gdm3" : no more complaint !!!

***********
Voila, simple dirty tweak, but works at least about the error; still need to know if apt-xapian-index works better or not.
The python3 port is still not complete, and python2.7 is still not completely removed. Will probably done with the 18.10 cycle.

If you want the same dirty tweak working on xenial, ..., then check the pathes first.

[Norbert (nrbrtx)]

For 16.04 LTS we can use

`sudo cp /usr/lib/python3.5/configparser.py /usr/lib/python3.5/ConfigParser.py`

And please note that problem is also seen on AskUbuntu - https://askubuntu.com/a/1049264/66509 .

But it is not enough - search is broken:

$ sudo rm -rf /var/lib/apt-xapian-index/
$ sudo apt-get update
$ sudo update-apt-xapian-index -f

$ axi-cache info
Main data directory: /var/lib/apt-xapian-index
Update timestamp: /var/lib/apt-xapian-index/update-timestamp (last updated: Sun Jun 24 13:20:47 2018)
Index location: /var/lib/apt-xapian-index/index (pointing to /var/cache/apt-xapian-index/index.2)
Documentation of index contents: /var/lib/apt-xapian-index/README
Documentation of available prefixes: /var/lib/apt-xapian-index/prefixes
Documentation of available values: /var/lib/apt-xapian-index/values
Plugin directory: /usr/share/apt-xapian-index/plugins
Plugin status:
  aliases enabled, up to date (114 days, 15:36:31.425668 older than index)
  app-install enabled, up to date (1018 days, 2:46:20.425668 older than index)
  apttags enabled, up to date (0:00:00 older than index)
  cataloged_time enabled, up to date (0:00:00 older than index)
  debtags disabled
  descriptions enabled, up to date (0:00:00 older than index)
  relations enabled, up to date (0:00:00 older than index)
  sections enabled, up to date (0:00:00 older than index)
  sizes enabled, up to date (0:00:00 older than index)
  template enabled, up to date
  translated-desc enabled, up to date (4 days, 4:00:57.425668 older than index)
Values:
  Value Code Provided by
  app-popcon 3 app-install
  catalogedtime 4 cataloged_time
  installedsize 1 sizes
  packagesize 2 sizes
  version 0 update-apt-xapian-index
Data sources:
  Source Description Used by
  /var/lib/apt-xapian-index/cataloged_times.p first-seen information for every package cataloged_time
  /usr/share/apt-xapian-index/aliases/popular-apps Aliases expanding names of popular applications aliases
  /var/cache/apt/pkgcache.bin APT index apttags, descriptions, relations, sections, sizes
  /usr/share/app-install/desktop/ .desktop files provided by app-install-data app-install

$ axi-cache search app
277 results found.
Results 1-20:
100% b'address-book-app-dbg' - (unknown by apt)
99% b'mediaplayer-app-autopilot' - (unknown by apt)
98% b'python-app-catalog-ui' - (unknown by apt)
98% b'ubuntu-app-test' - (unknown by apt)
98% b'gir1.2-ubuntu-app-launch-2' - (unknown by apt)
97% b'notes-app-autopilot' - (unknown by apt)
96% b'messaging-app-autopilot' - (unknown by apt)
96% b'camera-app-autopilot' - (unknown by apt)
95% b'dialer-app-autopilot' - (unknown by apt)
95% b'libghc-wai-app-file-cgi-doc' - (unknown by apt)
95% b'libubuntu-app-launch2-dev' - (unknown by apt)
94% b'reminders-app-autopilot' - (unknown by apt)
94% b'address-book-app-autopilot' - (unknown by apt)
94% b'libplack-app-proxy-perl' - (unknown by apt)
94% b'libubuntu-app-launch2' - (unknown by apt)
93% b'...

[Norbert (nrbrtx)]

Bug exists in 18.10 and 19.04.
Please fix it and provide SRU for LTS releases.

[Brian K. White (bkw777)]

Wow still present in 19.04

bkw@negre:~$ axi-cache
No module named 'ConfigParser': only help functions are implemented, for the sake of help2man

[Norbert (nrbrtx)]

Still present in 19.04.

[Norbert (nrbrtx)]

Still a problem on Eoan.

But works on the Debian Buster.

[Norbert (nrbrtx)]

@brian-murray

>added: rls-bb-notfixing rls-ee-notfixing

This should be fixed before 20.04 LTS release.
And again - it is broken only on Ubuntu. Debian is not affected.
Could please take care on this bug?

[Brian Murray (brian-murray)]

The Canonical Ubuntu Foundations team has made a decision that we won't be working on this particular bug given that it is about a package in universe.

[Norbert (nrbrtx)]

Ubuntu 19.10 is affected by this bug too.

This bug should be completely fixed before 20.04 LTS release.

Otherwise we will lose full-feature search in Synaptic and Muon.

[Norbert (nrbrtx)]

(if we remove `apt-xapian-index` package because of this bug)

[Seb Bonnard (sebma)]

@khurshid-alam
Hi, just noticed your [A hack to workaround error](
https://bugs.launchpad.net/ubuntu/+source/apt-xapian-index/+bug/1612948/comments/6) has a typo.
Can you please replace "ax-cache" by "axi-cache" and repost it ?

[Seb Bonnard (sebma)]

Here's the command I used to apply this patch :

$ sudo patch --verbose -d $(dirname $(which axi-cache)) < axi-cache.patch
Hmm... Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- a/usr/bin/axi-cache
|+++ b/usr/bin/axi-cache
--------------------------
patching file axi-cache
Using Plan A...
Hunk #1 succeeded at 47.
Hunk #2 succeeded at 115.
Hunk #3 succeeded at 171.
Hmm... Ignoring the trailing garbage.
done

[Carlo Vanini (silhusk)]

This bug has been fixed in version 0.50 of apt-xapian-index.
If you need a fix for the bug in previous versions of Ubuntu, please follow the instructions for "Requesting a Backport" at https://wiki.ubuntu.com/UbuntuBackports#Requesting_a_Backport.

The `configparser` import has been fixed in commit https://salsa.debian.org/debian/apt-xapian-index/-/commit/bcefb8ae63aed7926b60a85d540a095517e821f7
The output encoding has been fixed in commit https://salsa.debian.org/debian/apt-xapian-index/-/commit/9458bb5226d1ae0deb90587458cfb60d13f38fa4