No messages for Territorial Lord?

[Klaus Halfmann]

Played "Territorial Lord" with a recent dev-build. But I got no notifications about the territory, even After I conquered the whole map.

can someon confirm.

[SirVer (sirver)]

do you got a savegame shortly before the time runs out?

[SirVer (sirver)]

Also, which map did you use?

[GunChleoc (gunchleoc)]

Please check if this is a duplicate of https://bugs.launchpad.net/widelands/+bug/1512093, in which case this should already be fixed.

[GunChleoc (gunchleoc)]

BTW you need a new game to test, loading an older savegame will load the win condition script without the recent fix.

[Klaus Halfmann (klaus-halfmann)]

I meanwhile tried two of my homegrown maps, I did see any message there.
There first one was the "official" finish lakes, I think.

What Do I have to do to selfmade maps to make them "usable" again?.

I will try another one from a trunk build, next time.

[Klaus Halfmann (klaus-halfmann)]

I did _not_ see any meesages in my selfmade maps.

[kaputtnik (franku)]

I could confirm this. Attached a savegame with map "Glacier lake" where i 'think' conquered most of the map. No message appears. According to data/scripting/win_conditions/territorial_lord.lua there should be a massage: "%s owns more than half of the map’s area."

Also the messages window shows nothing related.

[kaputtnik (franku)]

Forgotton: Version was bzr8064

[GunChleoc (gunchleoc)]

I just played Last Bastion, once winning and once losing. I got the messages in both cases.

I do not get any messages for #7 though - maybe all that unusable, unconquered terrain is counted, so you don't reach 50%?

[kaputtnik (franku)]

Yes, i thought also on the "land counting algorithm". If it counts also unavailable land (f.e. unreachable mountainous area) some maps maybe couldn't be played with this win condition.

[kaputtnik (franku)]

I think it counts right: All water and acid terrains are imho not counted.

I have added a print statement to territorial_lord.lua which should output some related information:

print('player name, owns fields/from overall fields ', p.name, ",", _landsizes[p.number], "/", # fields)

The last one ( # fields) should be the overall amount of fields which could be conquered. This value divided by two is the half of the area which has to be conquered to show a message. GunChleoc is this the right statement to get the size of a table in lua (number of all entries, "fields" in this case)?

Attached is the modified lua file. Since a savegame does store also the code of a win condition, using this file works only with new started games. The output for map "Glacier Lake" is f.e. like:

player name, owns fields/from overall fields Spieler 1 , 271 / 7043
player name, owns fields/from overall fields Spieler 2 , 271 / 7043
[...]

The output get updated every 30 seconds.

[Klaus Halfmann (klaus-halfmann)]

Can I add this to a savegame, too?
I will try to, but this has to wait until the weekend I guess.

[GunChleoc (gunchleoc)]

Yes, # is correct to count the size of a Lua table.

The changed script can't be added to a savegame, because it's saved in binary format rather than as plaintext script files.

[kaputtnik (franku)]

No, this is afaik not possible. Somewhere in the savegame the complete code of the wincondition is stored. Check it out:

- rename the original lua file
- save the attached file from #11 to data/scripting/win_conditions
- start a new game with win condition Territorial Lord (you will see the output on console)
- save the game
- replace the new lua file with the old one (So no print statement is available)
- Load your previous saved game and you will see the output come every 30 seconds again

In case of Map Glacier Lake in know now why there is no message: All the snow on the mountains is Winter snow, which is arable and so it counts as land to be conquered. But most of it couldn't be conquered because the height of the fields is such different that on most places a building couldn't be build. The Value of the print output in #11 (7043) is just:

map size (96x80=7680) minus number of fields with water terrain.

[GunChleoc (gunchleoc)]

It's not a regression then, so I'm retargeting this. We need a more intelligent function then to take into account only fields where at least a small building, a port or a mine could be built if there were no immovables (e.g. trees/rocks) on it.

[Klaus Halfmann (klaus-halfmann)]

I played bzr8082[bug-1618547-wares-queue-order] with kapputniks channge now upto the bitter end
(Atlanters vs. Imperial un Twin Lagoos). what I got was this output

player name, owns fields/from overall fields Hasi50 , 5391 / 10089
player name, owns fields/from overall fields Computer 2 , 1934 / 10089

about 39 min later

player name, owns fields/from overall fields Hasi50 , 6049 / 10089
player name, owns fields/from overall fields Computer 2 , 1939 / 10089

about 1 and 1/2 hours later

player name, owns fields/from overall fields Hasi50 , 7331 / 10089
player name, owns fields/from overall fields Computer 2 , 1363 / 10089
[Host]: Received a connection request

almost done

player name, owns fields/from overall fields Hasi50 , 8392 10089
player name, owns fields/from overall fields Computer 2 , 408 10089

player name, owns fields/from overall fields Hasi50 , 8784 / 10089
player name, owns fields/from overall fields Computer 2 , 16 / 10089

But I got no message whatsoever.

As this seems kind of broken for R19 can we warn then user that this may fail?

I will try TerrtitorialTime now ...

[kaputtnik (franku)]

Klaus, i couldn't reproduce the values you have on map "Twin Lagoons", except the overall number. When i conquered most of the map i get:

player name, owns fields/from overall fields Player 1 , 4429 / 10089
player name, owns fields/from overall fields Player 2 , 1146 / 10089

Could you please attach a save game here? So we could compare the values on different systems. Attached is my save game.

[Klaus Halfmann (klaus-halfmann)]

Here is my autosave (playing in german...) some 30 minutes before I was done.

kaputtnick: Ill try your one and check the logs ...

Rememeber: I am using OSX, and I played in German, as atlanteers.

All of these may account to these differences, mmmh.

[Klaus Halfmann (klaus-halfmann)]

I was unable to load this file in my version of trunk bzr8076[trunk]
I loaded it then vial the commandline and got:

player name, owns fields/from overall fields Player 1 , 4429 / 10089
player name, owns fields/from overall fields Player 2 , 1146 / 10089

well you still have not found all the area I have seen, so this looks reasonable to me,

[kaputtnik (franku)]

Yes, my fault... wrong save game :-S

The newly attached save game is just the situation were i conquered more than half the available fields. You could see the message in the message window.

The main difference between your save game and mine is that you played multiplayer. I guess here is the culprit.

[Klaus Halfmann (klaus-halfmann)]

Checky my lua verison which my be he Problem, too:
Lua 5.3.3 Copyright (C) 1994-2016 Lua.org, PUC-Rio

will try your file tomorrow ...

[GunChleoc (gunchleoc)]

Lua 5.3.x is what we support now.

[kaputtnik (franku)]

I have created a simple new map for testing and tested also multiplayer with this map. Here all is fine, the status message appears.

Attached the multiplayer save game.

[Klaus Halfmann (klaus-halfmann)]

Hmm, maybe just the messages are lost as of the network play?

OK I loaded you game and Indeed see the message, in the logs I find:

player name, owns fields/from overall fields Player 1 , 5213 / 10089
player name, owns fields/from overall fields Player 2 , 1133 / 10089
TrainingSite::drop_stalled_soldiers: Kicking somebody out.
...
player name, owns fields/from overall fields Player 1 , 5393 / 10089
player name, owns fields/from overall fields Player 2 , 1133 / 10089

(This is no network game, sure ...)

Ahh, you already got all the Gold that was to be found at your base :-) ...

OK, I played now for some 20 Minutes , but got no message, will atatch my savegame now ...

[kaputtnik (franku)]

Hm, that's really strange: I don't get a message after loading my save game attached in #20 for more than 20 Minutes. So maybe loading a save game brake something?

Are the "20 minutes" related to in game time (affected by game speed) or real time?

[Klaus Halfmann (klaus-halfmann)]

I actually pyaed (I think) 30 Minutes, at normal speed, so this was way longer then then 20 minutes when the message should have appeard. I must try your special map (without save)...

[GunChleoc (gunchleoc)]

It's the gametime.

[Klaus Halfmann (klaus-halfmann)]

I was playing TerritorialLordTime now and found thta this is broken as well.
See #1622121, perhaps this is all the same Issue.

[Klaus Halfmann (klaus-halfmann)]

kapunik, I now played territorial_lord_newmap.wgf and everything was expected.
(The last messaeg in the news was in english, but everthing else was in german)

So we mut check in two directions:
 * Are Maps like "Glacier Lage" and "Twin Lagoons" broken?
 * Are savegames on OSX broken?

I will load that map again and just save it whithout much playing.

[Klaus Halfmann (klaus-halfmann)]

I placed two roads on the Map for an X ...

[SirVer (sirver)]

So, investigating running coroutines in Widelands is not super convenient right now, but it sorta works. I loaded the MoreThenHalf.wgf file and used this Lua script to investigate what is going on:

----
for k, entry in pairs(__coroutine_registry) do
   if tonumber(entry) == nil then
      print(tostring(entry), ":")
      local done = false
      for i=1,100 do
         if done then break end
         for j=1,100 do
            if not pcall(debug.getlocal, entry, i, j) then
               done = true
               break
            end
            l = debug.getlocal(entry, i, j)
            if l == nil then
               break
            end
            k, v = debug.getlocal(entry, i, j)
            print(("%s%q = %q"):format((" "):rep(i*3), k, tostring(v)))
            if (type(v) == "table") then
               for tk, tv in pairs(v) do
                  print(("%s%q = %q"):format((" "):rep((i+1)*3), tk, tostring(tv)))
               end
            end
         end
      end
   end
end

----
A few remarks to explain what is going on: the debug.* functions are provided by Lua to investigate a running callstack. It is cumbersome, but essentially this script lists all existing coroutines with their local variables (indented by stack depth) and their values. If I run this on the provided savegame I get:

thread: 0x7ff6d30f0188 :
   "time" = "30000"
      "plrs" = "table: 0x7ff6d123dd30"
         "1" = "Player(1)"
         "2" = "Player(2)"
      "fields" = "table: 0x7ff6d1236d70"
        ... lines omitted
      "map" = "table: 0x7ff6d30f1e00"
         "0" = "userdata: 0x7ff6d30f1e68"
      "currentcandidate" = "Player 1"
      "candidateisteam" = "false"
      "remaining_time" = "990"
      "teamnumbers" = "table: 0x7ff6d1236ef0"
      "_landsizes" = "table: 0x7ff6d1237700"
         "1" = "5895"
         "2" = "1101"
      "_calc_current_landsizes" = "function: 0x7ff6d1237fa0"
*** Ending Lua interpretation!

1) there is only one coroutine. This is clearly a bug, but also understandable:
      run(function()
         sleep(5000)
         check_player_defeated(plrs, lost_game.title, lost_game.body, wc_descname, wc_version)
      end)

never loops - so it will only check once for defeated players. So this is a bug in the win condition.

2) remaining_time is 990 seconds and never changes, the _landsizes table is regularly updated though.
3) _calc_current_landsizes is the only function that is listed in the context of this coroutine. Looking through the code I realize that all other functions are globals, i.e. not defined "local function()". Since some of them reference variables from the local frame (like remaining_time), this feels volatile - global funcitons referencing a local frame. I did not test yet if changing those functions to be local () is enough - it might be, because it strongly constraints which variables are available in the functions frame.
4) the logic of the win condition is pretty hard to follow - maybe there is just a plain old bug lurking in it.

[SirVer (sirver)]

We should maybe figure out if we can use something in http://lua-users.org/wiki/DetectingUndefinedVariables to enforce local variables and make use of accidental globals impossible.

[Klaus Halfmann (klaus-halfmann)]

SirVer: as of #1617576 I found that there is some differnce between saving stadanlone and networking games, no Idea if this count to this one as well.

Admitted: I ll need some lectures about this coroutine stuff before I will be able to grasp waht you wrote. But it sound like you know how to fix this ...

[SirVer (sirver)]

Klaus, I pushed a branch that might fix this. I am not sure though. This is the problem I found: Every win condition defines a function that gets executed as a coroutine (a coroutine is just a function that can sleep and be woken at the same point of execution later on). The territorial* ones defined global functions that reference local variables, and that might be a problem. Example:

include "scripting/coroutine.lua" -- for run and sleep
run(function()
    print("I am a the main coroutine of the win condition.")

    local i = 0
    function blub()
       i = i + 1
    end

    while true do
     print("i is", i)
     blub()
     sleep(5000)
    end
end)

If you launch the game this coroutine will wakeup every 5000 ms and execute what is inside the while true loop. And this will work nicely without saving and loading. The blub() function has a reference to the 'i' variable and will just increment it.

What happens on save and load though? The global Lua frame is saved before all coroutines frame. Since 'blub' is not defined as 'local function blub()' it is actually a global variable - and will be saved in the global frame. The variable 'i' though is local and will be saved with the reference frame of the coroutine.

On load, the 'blub' function will be restored into the global frame - but the local variable 'i' is only loaded once the coroutine contexts are loaded. So I am unsure what variable 'blub' is actually mutating after load.

I am unsure if this is really the (full) problem, but I attempted a fix in the linked branch and would appreciate testing.

[Klaus Halfmann (klaus-halfmann)]

That local_functions did the trick!
thnanks SirVer.

Ill try to cleanup the web of tickets and branches, later ,....

[GunChleoc (gunchleoc)]

Excellent :)

global being the default in Lua has kicked us in the butt multiple times now.

[GunChleoc (gunchleoc)]

Fixed in build19-rc1.