One wrong migration / migrations files in general

[kaputtnik]

I just found one wrong migration related to wlmaps, but it's not Critical.

First a few words about the migration files found in each app under the folder 'migrations':
Django provides a functionality to manage database related changes, like changing length of fields, adding fields to models (do you remember adding the 'hint' field to wlmaps?), or changing field parameters. This works only if a folder 'migrations' (with __init__.py) exists.
The state before the merge of the django1_8 was that we hadn't such a folder for our own apps. F.e in case of the 'hint' field of wlmaps, we had to modify the related table in the database directly, and had to be sure that the appropriate model of wlmaps reflect this changes. With the migrations folder the work were much easier: Modify the Django model, ran './manage.py makemigrations' and './manage.py migrate'. Django then will create another file in the migration folder with the needed command and apply the changes to the database table for us. Much convenient. Nevertheless there are some things to pay attention on. but those are specific to some type of fields.

The wrong migration file and why it's not critical:
The wrong file wlmaps/migrations/0001initial.py contains two statements providing my own folder structure, f.e.:

('file', models.FileField(upload_to=b'/home/kaputtnik/wl_django1_8/code/widelands/media//wlmaps/maps/', verbose_name=b'Mapfile')),

This is totally wrong on the server... but the good news: Because of running './manage.py migrate --fake-initial' the initial files are not processed in real (applied to the database tables). They are only marked as applied and the old table schema is not affected.
Nevertheless this statement is wrong if someone else want to set up the website locally and ran './manage.py migrate'. If he/she wants testing uploading maps this will fail...

The reason why this was happened:
By investigating this failure i found that the underlying code was not the code Django expects: The 'file' field in wlmaps/models.py defines the option 'upload_to' with the whole path of MEDIA_ROOT + '/wlmaps/maps/'. This results in huge strings for the file field. I have looked into the table and found for example
"/var/www/django_projects/wlwebsite/code/widelands/media//wlmaps/maps/Untergegangene Siedlung.wmf"
which is 98 chars. The field is defined with a length of 100 chars...

The solution is now to give Django what it expects: Only the additional path to MEDIA_ROOT, in this case: 'wlmaps/maps' (without the preceding and leading slashes). The value of the setting MEDIA_ROOT is applied by Django automatically. I've looked into the other models where file upload is possible and there all is fine. So it's only wlmaps which is problematic.

Next week i want to provide a branch with the needed changes to fix this.

Of course i had to take look at all the migrations files, but unfortunately i didn't. I am sorry about that...

[GunChleoc (gunchleoc)]

This kind of thing just happens when you do huge changes - and since it worked, nobody noticed the bug for a while ;)

[kaputtnik (franku)]

As mostly after clicking "Post Comment" my Brain is thinking again and has come to the conclusion that the wrong migration in this case does NOT effect other installations. And i have tested this on a second account without any errors.

But the building of the huge strings for the maps in the database should be fixed anyway...

[GunChleoc (gunchleoc)]

> But the building of the huge strings for the maps in the database should be fixed anyway...

+1

[kaputtnik (franku)]

Hm.. there is also a mismatch where the minimaps are stored. Before 03.06.1014 they are stored in 'media/wlmaps/minimap/' and after then they are stored in 'media/wlmaps/maps/' (together with the wmf files). This is no problem but a question of style:

Should the minimaps be stored in the same folder where the mapfile is stored?

From my point of view the images and data files should be differentiated, but in this case they belong together. I am unsure. What do you think?

Just asking because i am working on the maps and could also change the current save path for minimaps. :-)

[GunChleoc (gunchleoc)]

Since we can potentially get a lot of files, having separate directories will make listing the dirs a lot faster, so I am in favour of splitting.