improve south support

[Daniel Abel]

(is there a way for me to mark this as a whishlist bug?)

south is a widely used database migration framework for django. It would be nice if sortedm2m would support south out-of-the-box. This would involve adding a bit of code to fields.py (introspection rules south uses), and possibly fixing bugs.

1) south needs introspection rules (see http://south.aeracode.org/docs/customfields.html#extending-introspection), the following appear to work (with south 0.7.1 and trunk version of sortedm2m):
---------------------------------------
try:
    from south.modelsinspector import add_introspection_rules
    add_introspection_rules([
        (
          [SortedManyToManyField],
          [],
          {'sorted': ['sorted', {'default': True}],
           },
        ),
        ], ["^sortedm2m\.fields\.SortedManyToManyField"])
except ImportError:
    pass
---------------------------------------
I think the best place to put this would be the end of fields.py

2) The following models.py works with the introspection rules above:
------------------------------
    class Book(models.Model):
        title = models.CharField(max_length=50)

    class Shelf(models.Model):
        books = SortedManyToManyField(Book)
------------------------------

However, putting the two models in reverse order, like this:
------------------------------
    class Shelf(models.Model):
        books = SortedManyToManyField('Book')

    class Book(models.Model):
        title = models.CharField(max_length=50)
------------------------------

Triggers the following errors when the migation is run:
------------------------------
Error: One or more models did not validate:
myapp.shelf: 'books' has an m2m relation with model <class 'myapp.models.Book'>, which has either not been installed or is abstract.
myapp.shelf: 'books' specifies an m2m relation through model <class '.myapp_Shelf_books'>, which has not been installed.
myapp.shelf: Accessor for m2m field 'books' clashes with related m2m field 'Book.shelf_set'. Add a related_name argument to the definition for 'books'.
myapp.shelf: 'books' has an m2m relation with model <class 'myapp.models.Book'>, which has either not been installed or is abstract.
myapp.shelf: 'books' specifies an m2m relation through model <class '.myapp_Shelf_books'>, which has not been installed.
myapp.shelf: The model Shelf has two manually-defined m2m relations through the model myapp_Shelf_books, which is not permitted. Please consider using an extra field on your intermediary model instead.
myapp.shelf: Accessor for m2m field 'books' clashes with related m2m field 'Book.shelf_set'. Add a related_name argument to the definition for 'books'.
------------------------------

This might be caused by a problem in south, I didn't investigate further. I did, however try an other workaround for this (reordering the model classes like in the first, working, example being one possibility): First creating a migration which adds the Book model, and adding the Shelf model in a second migration. In this case, I got the following exception when the migration was created:
------------------------------
$ python manage.py schemamigration myapp add_shelf --auto
<class 'myapp.models.Book'> <sortedm2m.fields.SortedManyToManyField object at 0x18e8390> <class 'myapp.models.Shelf'> books
<class 'myapp.models.Book'> <sortedm2m.fields.SortedManyToManyField object at 0x18e8390> <class 'myapp.models.Shelf'> books
 + Added model myapp.Shelf
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_manager(settings)
  File "/mnt/data/abeld/WORK/maven/FirmNetOnline/sortedm2m_bug/env/lib/python2.6/site-packages/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/mnt/data/abeld/WORK/maven/FirmNetOnline/sortedm2m_bug/env/lib/python2.6/site-packages/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/mnt/data/abeld/WORK/maven/FirmNetOnline/sortedm2m_bug/env/lib/python2.6/site-packages/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/mnt/data/abeld/WORK/maven/FirmNetOnline/sortedm2m_bug/env/lib/python2.6/site-packages/django/core/management/base.py", line 218, in execute
    output = self.handle(*args, **options)
  File "/mnt/data/abeld/WORK/maven/FirmNetOnline/sortedm2m_bug/env/lib/python2.6/site-packages/south/management/commands/schemamigration.py", line 143, in handle
    action.add_forwards(forwards_actions)
  File "/mnt/data/abeld/WORK/maven/FirmNetOnline/sortedm2m_bug/env/lib/python2.6/site-packages/south/creator/actions.py", line 29, in add_forwards
    forwards.append(self.forwards_code())
  File "/mnt/data/abeld/WORK/maven/FirmNetOnline/sortedm2m_bug/env/lib/python2.6/site-packages/south/creator/actions.py", line 428, in forwards_code
    "left_field": self.field.m2m_column_name()[:-3], # Remove the _id part
TypeError: 'NoneType' object is unsubscriptable
------------------------------

[Daniel Abel (abeld)]

From my previous comment:
> 1) south needs introspection rules (see http://south.aeracode.org/docs/customfields.html#extending-
> introspection), the following appear to work (with south 0.7.1 and trunk version of sortedm2m)

Actually, it doesn't work properly: the migration generated by south contains the following:
---------------------
        # Adding M2M table for field books on 'Shelf'
        db.create_table('myapp_shelf_books', (
            ('id', models.AutoField(verbose_name='ID', primary_key=True, auto_created=True)),
            ('shelf', models.ForeignKey(orm['myapp.shelf'], null=False)),
            ('book', models.ForeignKey(orm['myapp.book'], null=False))
        ))
---------------------
Which means that the sort_value column is not recognized by south, which will cause problems later on, since the column won't be created in the database

[Daniel Abel (abeld)]

Experimenting with possible workaround, I tried explicitly specifying the intermediary model, in the hopes that south will pick this up. However, it appears that sortedm2m does not allow the intermediary model to be overriden: even when some model is passed in in the 'through' argument, the auto-generated intermediary model is created and the passed-in model is overriden by it. Using the following models.py:
------------------------------
     class Book(models.Model):
         title = models.CharField(max_length=50)

     class Shelf(models.Model):
         books = SortedManyToManyField(Book, through='BookOnShelf')

     class BookOnShelf(models.Model):
         shelf = models.ForeignKey(Shelf)
         book = models.ForeignKey(Book)
         sort_value = models.IntegerField()
------------------------------
I get a myapp_shelf_books table and a myapp_bookonshelf table.

It seems to me that this is a seperate bug in itself: although SortedManyToManyField has a 'through' argument (inherited from ManyToManyField), it cannot be used the way it can be for ManyToManyField.

It appears, however, that another (ugly) workaround works for my original problem, for making sortedm2m work with south: adding the following by hand to the tuple describing the fields in "db.create_table('myapp_shelf_books'...":
-------------------------------
('sort_value', self.gf('django.db.models.fields.IntegerField')()),
-------------------------------
appears to result in the correct database structure when the migration is applied.

Unfortunately I don't know how to tell south to add this field. For now, having to fix the migrations by hand is a usable workaround for me