natural sorting in the page index

[milk2cups]

When I use Alt-PageDown and Alt-PageUp to enact "Next in index" and "Previous in index" in the Go menu, I will go through Part1, Part10, Part11, Part12, Part2, Part3, ..., Part8, Part9.

This behavior is annoying. I want the order to be: Part1, Part2, ..., Part8, Part9, Part10, Part11, Part12.

I do not want to change the pages' names to Part01, Part02 etc not only because this is ugly but also because for example, the initial expectation might be 42 parts in total. However, I might end up with more than 1000 parts.

Again, thanks for your effort!

I am using zim 0.27.

[dotancohen (dotancohen)]

This is called "natural sorting" and so far as I understand there is no native GTK implementation of it yet. Qt only got natural sorting in Qt 4, released last year. Search on the GTK bugzilla and if you don't find a natural sorting feature request then file one. In any case, post the bug number here so that we can keep abreast.

[Jaap Karssenberg (jaap.karssenberg)]

What you want is numerical instead of purely alphabetical sorting. Can be done, just wondering if there are any cases where this would go wrong.

Also see bug #278417 which would allow setting the correct order manually.

[milk2cups (milk2cups)]

But I don't think it's my position to say whether this should be a gui lib
feature or simply an app feature. A user should not be expected to dive
that deep into the implementation details. Sorry.

On Mon, Dec 1, 2008 at 5:57 PM, dotancohen <email address hidden> wrote:

> This is called "natural sorting" and so far as I understand there is no
> native GTK implementation of it yet. Qt only got natural sorting in Qt
> 4, released last year. Search on the GTK bugzilla and if you don't find
> a natural sorting feature request then file one. In any case, post the
> bug number here so that we can keep abreast.
>
> --
> next in index: 1 2 3 or 01 02 03
> https://bugs.launchpad.net/bugs/303657
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in Zim desktop wiki: New
>
> Bug description:
> When I use Alt-PageDown and Alt-PageUp to enact "Next in index" and
> "Previous in index" in the Go menu, I will go through Part1, Part10, Part11,
> Part12, Part2, Part3, ..., Part8, Part9.
>
> This behavior is annoying. I want the order to be: Part1, Part2, ...,
> Part8, Part9, Part10, Part11, Part12.
>
> I do not want to change the pages' names to Part01, Part02 etc not only
> because this is ugly but also because for example, the initial expectation
> might be 42 parts in total. However, I might end up with more than 1000
> parts.
>
> Again, thanks for your effort!
>
> I am using zim 0.27.
>

[milk2cups (milk2cups)]

#278417 is interesting.

My suggestion is to put the sorting information in the page itself. I mean,
the zim pages are stored in a format like an email message now. I think it
is good. I suggest you add a header there such as: Next: PartX.

Then you can do caching in anyway you deem as efficient.

And when the user has not adjust the order in the GUI, the Next header can
have default values. When it's the default value, the GUI shall display a
sensible default. I mean, natural numerical ordering. ;)

Anyway, my 2c.

Thank you again. And again. :)
I like zim!

On Mon, Dec 1, 2008 at 6:43 PM, Jaap Karssenberg <email address hidden> wrote:

> What you want is numerical instead of purely alphabetical sorting. Can
> be done, just wondering if there are any cases where this would go
> wrong.
>
> Also see bug #278417 which would allow setting the correct order
> manually.
>
> ** Changed in: zim
> Importance: Undecided => Wishlist
> Status: New => Confirmed
>
> --
> next in index: 1 2 3 or 01 02 03
> https://bugs.launchpad.net/bugs/303657
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in Zim desktop wiki: Confirmed
>
> Bug description:
> When I use Alt-PageDown and Alt-PageUp to enact "Next in index" and
> "Previous in index" in the Go menu, I will go through Part1, Part10, Part11,
> Part12, Part2, Part3, ..., Part8, Part9.
>
> This behavior is annoying. I want the order to be: Part1, Part2, ...,
> Part8, Part9, Part10, Part11, Part12.
>
> I do not want to change the pages' names to Part01, Part02 etc not only
> because this is ugly but also because for example, the initial expectation
> might be 42 parts in total. However, I might end up with more than 1000
> parts.
>
> Again, thanks for your effort!
>
> I am using zim 0.27.
>

[Jaap Karssenberg (jaap.karssenberg)]

Needs sorting function / library which can be used when populating the index.

[Lisa (syn-shainer)]

Hi,
I would like to work on this. I think milk2cups' idea is good. The sorting function can be called when the user decides to change the sorting type (through a menu voice o dragging the pages around) and the Next fields will be updated.

Just a question: the natural sorting should be applied even when the numbers are in a different position, like at the beginning of the filename or in the middle?

[milk2cups (milk2cups)]

On Wed, Mar 23, 2011 at 5:09 AM, Lisa <email address hidden> wrote:
> Hi,
> I would like to work on this. I think milk2cups' idea is good. The sorting function can be called when the user decides to change the sorting type (through a menu voice o dragging the pages around) and the Next fields will be updated.
>
> Just a question: the natural sorting should be applied even when the
> numbers are in a different position, like at the beginning of the
> filename or in the middle?

I suggest you only do the obvious ordering, and do nothing if the
situation becomes tricky.
i.e., part1, part2, ..., part10. but do not try too hard to figure out
the ordering of part1x and part_2aaa.
Also, I think there're some conventions regarding software version
numbers, e.g., py3alpha, py3beta, py3rc, etc., you might want to take
that into consideration as well.

[Jaap Karssenberg (jaap.karssenberg)]

On Wed, Mar 23, 2011 at 9:16 AM, milk2cups <email address hidden>wrote:

> On Wed, Mar 23, 2011 at 5:09 AM, Lisa <email address hidden> wrote:
> > Just a question: the natural sorting should be applied even when the
> > numbers are in a different position, like at the beginning of the
> > filename or in the middle?
>
> I suggest you only do the obvious ordering, and do nothing if the
> situation becomes tricky.
> i.e., part1, part2, ..., part10. but do not try too hard to figure out
> the ordering of part1x and part_2aaa.
>

This can be easily achieved by splitting the name in numeric and non-numeric
parts and then sort based on that list.

> Also, I think there're some conventions regarding software version
> numbers, e.g., py3alpha, py3beta, py3rc, etc., you might want to take
> that into consideration as well.
>

I would advise against this as version numbering is highly inconsistent
across software projects. Only thing like a spec in this area is the
numbering of debian package version numbers. Fortunately that is mostly
based on the trick mentioned above: split numeric and non numeric parts +
splitting on some special characters like "-", "." and ":".

-- Jaap

[milk2cups (milk2cups)]

On Wed, Mar 23, 2011 at 9:30 PM, Jaap Karssenberg
<email address hidden> wrote:
> On Wed, Mar 23, 2011 at 9:16 AM, milk2cups
> <email address hidden>wrote:
>
>> On Wed, Mar 23, 2011 at 5:09 AM, Lisa <email address hidden> wrote:
>> > Just a question: the natural sorting should be applied even when the
>> > numbers are in a different position, like at the beginning of the
>> > filename or in the middle?
>>
>> I suggest you only do the obvious ordering, and do nothing if the
>> situation becomes tricky.
>> i.e., part1, part2, ..., part10. but do not try too hard to figure out
>> the ordering of part1x and part_2aaa.
>>
>
> This can be easily achieved by splitting the name in numeric and non-numeric
> parts and then sort based on that list.

The problem is that you don't necessarily know what the user's intention is

[Lisa (syn-shainer)]

Well, if the user selects the natural sorting for his pages I think all the numbers in every position should be naturally sorted.

IMHO, The conventions for software projects number would make the code too much complex. It can be useful for software version numbers like 2.3.4, which are very common in lots of projects, but I don't see it as a very important feature to be added.

Lisa

[Lisa (syn-shainer)]

Patch coming!

Just two notes:
1) I'm not very sure about how the Home page is handled, so I set manually its position to 1 when the first customized page is created. Sure there's a better method. Look for a FIXME comment in the code to find this part.
2) I added a new column in the pages table, so existent databases should issue an ALTER TABLE command or re-initialize the database.

[Jaap Karssenberg (jaap.karssenberg)]

On Fri, Apr 1, 2011 at 2:18 PM, Lisa <email address hidden> wrote:

> Patch coming!
>
> Just two notes:
> 1) I'm not very sure about how the Home page is handled, so I set manually
> its position to 1 when the first customized page is created. Sure there's a
> better method. Look for a FIXME comment in the code to find this part.
>

Strange, Home should not be special in any way in the index.

> 2) I added a new column in the pages table, so existent databases should
> issue an ALTER TABLE command or re-initialize the database.
>

There is a database version in the code, when that is increased database is
re-initialized automatically. So no problem there.

** Patch added: "Bugfix-#303657.patch"
>
> https://bugs.launchpad.net/zim/+bug/303657/+attachment/1963415/+files/Bugfix-%23303657.patch
>

Had a quick look and looks solid, will test it out early next week.

Thanks!

Jaap

[ceg (ceg)]

Does the patch also allow the reordering as in Bug #278417 ?

[Jaap Karssenberg (jaap.karssenberg)]

On Mon, Apr 11, 2011 at 12:41 PM, ceg <email address hidden> wrote:

> Does the patch also allow the reordering as in Bug #278417 ?
>

No. That is a different request.

-- Jaap

[ceg (ceg)]

Sorry if I got confused, I only meant to ask if the metadata format supports the reordering.

Comment #4 milk2cups :
> I suggest you add a header there such as: Next: PartX.

Comment #6 Lisa:
> I would like to work on this. I think milk2cups' idea is good.

That suggested a chain references in the pages, so it should be possible to reorder, right? (currently no GUI of course)

Or was natural sorting done differently, and above comment was only related to the other reordering issue and just unrelated here?

[Jaap Karssenberg (jaap.karssenberg)]

On Mon, Apr 11, 2011 at 2:37 PM, ceg <email address hidden> wrote:

> Sorry if I got confused, I only meant to ask if the metadata format
> supports the reordering.
>
> Comment #4 milk2cups :
> > I suggest you add a header there such as: Next: PartX.
>
> Comment #6 Lisa:
> > I would like to work on this. I think milk2cups' idea is good.
>
> That suggested a chain references in the pages, so it should be possible
> to reorder, right? (currently no GUI of course)
>
> Or was natural sorting done differently, and above comment was only
> related to the other reordering issue and just unrelated here?
>

There is a index in the database giving the sorting order. This is not
enough for re-ordering since the index is just cache (it needs to be
possible to rebuild it from the data). So re-ordering probably needs it's
own index.

Please do not discuss further here, but on the ticket related to
re-ordering.

-- Jaap

[dotancohen (dotancohen)]

Does Lisa's patch apply to the current Zim? This would be a nice complimentary feature for the next Zim release, which is shaping up to be a very mature application.

Thanks.

[Jaap Karssenberg (jaap.karssenberg)]

On Tue, Jan 10, 2012 at 11:18 AM, dotancohen <email address hidden> wrote:
> Does Lisa's patch apply to the current Zim? This would be a nice
> complimentary feature for the next Zim release, which is shaping up to
> be a very mature application.

No. I do have a branch based on this patch with enhancements, but
still issues with performance. Also found that my code for the index
is messy, so need to refactor before I can add this feature.

-- Jaap

[Jaap Karssenberg (jaap.karssenberg)]

On Tue, Jan 10, 2012 at 1:46 PM, Jaap Karssenberg
<email address hidden> wrote:
> On Tue, Jan 10, 2012 at 11:18 AM, dotancohen <email address hidden> wrote:
>> Does Lisa's patch apply to the current Zim? This would be a nice
>> complimentary feature for the next Zim release, which is shaping up to
>> be a very mature application.
>
> No. I do have a branch based on this patch with enhancements, but
> still issues with performance. Also found that my code for the index
> is messy, so need to refactor before I can add this feature.

P.S. have an alternative idea for fixing this that will also take into
account collation of non-ascii characters based on current locale.

-- Jaap

[dotancohen (dotancohen)]

For what it's worth, I use many non-ASCII names for my pages. I notice that they collate from A to Z first, then from א to ת.‎ That is the correct order for both languages.

[Jaap Karssenberg (jaap.karssenberg)]

On Tue, Jan 10, 2012 at 2:16 PM, dotancohen <email address hidden> wrote:
> For what it's worth, I use many non-ASCII names for my pages. I notice
> that they collate from A to Z first, then from א to ת.‎ That is the
> correct order for both languages.

Yes, that makes sense for two complete alphabets. But for locales that
use an extended ascii set, like some eastern European locales, the
extended characters should be mixed with the ascii characters. E.g.
"a" with an accent on it should still sort under "a".

Regards,

Jaap

[Jaap Karssenberg (jaap.karssenberg)]

Committed fix in rev476, taking care of natural sort and collating locale aware. Needs testing, so please check this version.

-- Jaap

[Jaap Karssenberg (jaap.karssenberg)]

Fixed in zim 0.55