Convert map/savegame files to lua

[Benedikt Straub]

We should save map files and savegames as lua files. This will make them human-readable and make it possible to edit them manually.

A map or savegame could be one large table with subtables for terrains, resources, immovables etc that map every coordinate point to the desired value.
For scenarios, the lua file could contain a function, and the mapfile-table has an entry that points to the function.
The minimap and other images would have to be stored separately somewhere.

new_map_description{
   name = _"Checkmate",
   size = {96, 96},
   resources = {
      {20, 10} = {"coal", 10},
      …
   },
   terrains = {
      {0, 0, "r"} = "summer_water",
      {0, 0, "d"} = "summer_meadow2",
      …
   },
   …,
   game_data = { -- savegames only
      players = {"My Name", "Your Name"},
      tribes = {"empire", "atlanteans"},
      field_owners = {
         {0, 0} = 1,
         …
      },
      binary = {
         -- lots of serials for economy state, AI-persistance etc, doesn´t need to be human-readable
      },
      …
   },
   scenario = my_main_function_which_i_defined_above, -- or nil
   minimap = "data/maps/pics/checkmate.png",
   …
}

[kaputtnik (franku)]

A save game is a saved game. Why should it be possible to tweak the saved game?

The only thing in could think of is to manipulate it. Would this be good goal?

[GunChleoc (gunchleoc)]

I had been thinking about this too, to get rid of the Profile class and make everything Lua.

There is the point that kaputtnik raised though - we have binary packets that can't be manipulated. Map compatibility is another issue - we would end up with 2 code paths. We already have 2 code paths for loading bobs, for example. We shouldn't add a third.

What we could indeed still convert to Lua is:

1. campaigns.conf and tutorials.conf. Already done: https://code.launchpad.net/~widelands-dev/widelands/campaignselect_box

2. The user options (.widelands/config)

[Benedikt Straub (nordfriese)]

You can already manipulate your game a bit by deleting individual packets ;)

Manually editing savegames can be very useful for debugging. It´d be possible to create a testcase for any bug you want to examine without having to bother with extensive scripting or playing a long game. And you could analyze everything about a saved game except internal serials and stuff.
Also, I consider the current savegame directories rather large. One file per savegame would be more handy.

But if you want to make sure players can't manipulate a savegame, it could be implemented that the savehandler calculates some special and complicated kind of checksum for the savegame (which is stored in the file); on loading the savegame, Widelands refuses to load a game if the saved checksum and the newly calculated one don't match, unless you start WL with a special --permit-modified-savegames option (for debugging). It would be very tedious for players to recalculate the checksum after editing the file.

[Benedikt Straub (nordfriese)]

OK valid point about map compatibility. So when implementing this, we´d have to keep the current loader and the s2-maploader around (though we could get rid of the current savers).
Just a brief question, we frequently break savegame compatibility, so why not break map compatibility as well once? After this change, we could put an announcement on the website: "Save your old maps anew please, compatibility for the old map format will be removed for Build 22", and a warning on the map-select screen so everyone will notice.

[GunChleoc (gunchleoc)]

We only break savegame compatibility when we absolutely have to, and we have never, ever broken map compatibility. Map compatibility has always been a feature, and we might have a rebellion on our hands if we remove it.

For scenario testing, I often hack the variables in the scenario, e.g. set water rising to true in the Atlantean scenario.

[GunChleoc (gunchleoc)]

Another argument against: Lua Tables need a lot of memory, and loading savegames will become a lot slower too.

[GunChleoc (gunchleoc)]

Migrated to https://github.com/widelands/widelands/issues/3316