Too high ware preciousness can hurt AI

[TiborB]

Currently each ware has a preciousness defined in init.lua and AI is using it when making decisions what buildings to build.

However late bug 1767187 revealed that too hing number (254 in the case of bug) badly confused AI into insane building of metalworks. So the conclusion is that upper limit for preciousness is needed.

I want to propose value 25, that would be hardcoded and if exceeded throw an exemption.

Note that normal players would never see this error, only if somebody is fiddling with init.lua files.

[hessenfarmer (stephan-lutz)]

As already stated I think 25 is far to less of a value. For the beginning I think 100 would be fine.

[GunChleoc (gunchleoc)]

What happened in the mentioned bug is that a number was not copied over during an engine change, and therefore missing. This is now verified during tribe loading and will not happen again.

So, I'm not sure if we really need this. We haven't had a case of somebody editing in a number that is too high yet. How high are the chances really that somebody will edit an init.lua file and enter an insensible value on purpose?

This will need to be documented really well too, otherwise somebody playing in init.lua will wonder why a value change has no effect. If we should go for a restriction here and implement this bug, it absolutely needs to happen on the tribe loading side.

At the moment, we don't have any information on the value range for preciousness, so that should be added to the documentation in any case.

https://wl.widelands.org/docs/wl/autogen_wares_lua_tribes_units_init/

[TiborB (tiborb95)]

Highest used preciousness is 14 now. And I think we should not go over 20 or 25. The minimum we can do is mention this range in documentation.

Additionally we can implement two limits:
- hard (100 as hessenfarmer proposed) when the game would fail to load
- soft (value to be mentioned in documentation) which would only print warning into terminal output.

But I agree this is not urgent issue...

[ypopezios (ypopezios)]

Such values should be the result of calculations. If given some data, I could try producing some formulas.

[TiborB (tiborb95)]

The problem is that genetic algorithm is involved here so it is difficult to come up with "data"

[ypopezios (ypopezios)]

Data don't have to translate into specific things in the game. I can use the same numbers that you used to come with 14 or 25. Actually, anything you can provide me as input I can probably express it with numbers, in order to produce a non-arbitrary value.

[TiborB (tiborb95)]

It is only observation - when preciousness was 255 by mistake, it used to build many buildings of that type that were not utilized and never promoted (lack of experiences for the worker). So I dont have hard numbers..

[ypopezios (ypopezios)]

So you tried various numbers up to 255 until observing that 14 works best? Shouldn't such kind of observations and number-finding be done by training alone? Cause if the AI has to base its behavior on arbitrary numbers provided by human intuition, then it will never become competitive (unless maybe by mistake, which is highly doubtful).

[TiborB (tiborb95)]

No, I did not do any thorough testing.

Preciousness affects what will be built sooner and more. So this is really based only on intuition. But you have good comment - perhaps we should get rid completely of this and let AI figure out.

But this is the same old story - we dont have enough CPU time for training so we are using such "hacks" to help the AI.

But as I said, you made good point..

[ypopezios (ypopezios)]

Then let me try the inverse route: Could you give me some output data, from the things preciousness affects?

[TiborB (tiborb95)]

Well, the output can be number of metalworks depending on preciousness. I saw a game with 8 metalworks and building new one. Of course most of them idle (<10%).

But I have no statistics that I can present to you.

[ypopezios (ypopezios)]

No need for statistics at this point. If I prepare a formula of this form:

preciousness = f(number of relevant buildings, productivity of those buildings)

would you be able to feed that to the AI, so as it has a dynamic value of preciousness to guide its building decisions?

[TiborB (tiborb95)]

This is wrong approach. Genetic algorithm should consider count of buildings and their current utilization. And it tries to do it.

Preciousness should be some hint for AI (fixed one, as it is now), or not be at all.

[ypopezios (ypopezios)]

I thought that we were in the realm of hacks here. Current hack isn't less wrong an approach than mine. The question is: Until we get enough CPU time for training, do we want a better AI or not?

[TiborB (tiborb95)]

Well, here we discuss the values of preciousness in init files.
If you suggest scrapping them, this should belong under different topic.

But if it will not take much of time (mine), we can try it in separate branch...

[ypopezios (ypopezios)]

Maybe put a "genetic algorithm" tag to some bugs, so as I don't suggest alternatives for them. And maybe we could provide multiple versions of AI, one with pure genetics, one with no genetics at all, and one with a mix.

[TiborB (tiborb95)]

I think you can suggest new preciousness without even be aware of how are they used under the hood...

Also for alternative AI - this is not new idea, I see two obstacles:

-somebody have to rework engine to be able to pick from multiple AIs
-for any new AI we would need a person who would commit to take care of it - can be thousands of hours over couple of years

[ypopezios (ypopezios)]

> I think you can suggest new preciousness without
> even be aware of how are they used under the hood...

How? By making a random guess? And what about future changes which may affect preciousness? Won't they make that number wrong again?

I already see the day when someone will suggest a great idea, but they will tell him: "Forget it, cause we will have to retrain the AI all over again." With such a dark future, maybe a couple of years are worthy an investment...

[TiborB (tiborb95)]

Genetic algorithm is too complex to give you an formula. This is like in medicine, you cannot calculate the effectiveness of a new drug, but must do (clinic) tests :(

I am not against a great ideas, but I dont think that this one is great idea. But show me the code, show results of your tests and we will see.

I had so many great ideas that did not work at the end and I spent too much time on them.

[ypopezios (ypopezios)]

That was a quick idea, about a simple change for a small improvement. As about code, I'll stick with economy for now. If sometime in the future flags start using reliable preciousness values to decide priority of serving wares, the AI may like to have a look...

[hessenfarmer (stephan-lutz)]

Hi,

did not see that this discussion went on. During my trials to balance the frisians and to make the AI handle them properly I recognized some behaviours which led me to propose new preciousness for some Frisian wares.
Part of the problem is that not everything in the AI is controlled by genetic algorithm. The building algorithm, as far as I understood (experienced) it, decides which wares are needed most. Therefore it considers stocklevel and preciousness of the ware for the relevant tribe. So in theory I think that preciousness should reflect (more the less) the ratio of the necessary buildings to run an economy that supports the constant training of fully promoted soldiers.
So my values were intuitively set to get into this direction and this showed some promising results. However they were biased by the genetic algorithm values interfering with this process.
As the next step in balancing would be to have the numbers of buildings or exactly this economy. Perhaps we could review the preciousness under this context as well.
If my understanding is right we will have the ideal ratios in preciousness and the genetic algorithm adds some "character" to the AI.

[TiborB (tiborb95)]

hessenfarmer,

if you have suggestion for different numbers of preciousness - I am fine with this. I can then have some training running with these numbers, this is the least thing I can do.

Genetic algorithm takes more things into account so preciousness does not reflect directly to some intended output...

[ypopezios (ypopezios)]

Unlike the currently intuitive/arbitrary value of preciousness, stocklevel is an objective number. Could you point me to the part of the code where stocklevel is considered in deciding what to build next?

In general, formulas could be used to produce some meaningful ranges of numbers (protecting the AI from making particularly stupid choices), while the genetic algorithm could be used to pick the precise number within each of those ranges (allowing the AI to develop its own character).

[TiborB (tiborb95)]

Hi, look into this funtion:
https://bazaar.launchpad.net/~widelands-dev/widelands/trunk/view/head:/src/ai/defaultai.cc#L4841
it returns one of these states: kForced, kNeeded, kNotNeeded, kNotBuildable, kAllowed, kNeededPending, kForbidden - should be selfexplanatory :)

just search for stocklevel - there is not a single place, it depends also on type of building

[hessenfarmer (stephan-lutz)]

@ Tibor:
I don't have such numbers yet. But this is one of my "things on the list"

I know that there is no direct or linear effect. My explanation was just to describe the the general concept of my understanding what preciousness does.

@ ypopezios:
It might not be totally objective yet. But stocklevel shouldn't be the only value to decide on, cause the necessary amount for wares is different from ware to ware. And preciousness is describing which wares you need more of and which you need less of.

[ypopezios (ypopezios)]

I went through the code. Turns out that preciousness is just one instance in a code full of arbitrary values, most of which are not documented. Moreover, they are combined into countless lines of structureless code, which is almost unreadable in the large scale. And from what I can tell, if I was to make some change to it, it would be close to impossible to test its results. The overall effort would be a waste of time.

The positive thing is that now I'm fully aware of why Widelands' AI has major chronic weaknesses. By the way, genetics is neither the problem nor the solution. It could be useful if properly applied, but right now it contributes more to the complexity of the code than to the strength of the AI. If Tibor is willing to closely cooperate, I could consider spending a couple of years, the first one not making any functional changes, just rewriting and documenting the existing code. Otherwise, I have no hope that the AI will ever present any challenge to the players.

[TiborB (tiborb95)]

Well, the code is quite structured, so if you want to focus to some partial aspect of AI (placing of new buildings, attacking, road making, seafaring) you can look up relevant functions.
If you look at defaultai.cc in build 19 https://bazaar.launchpad.net/~widelands-dev/widelands/b19/view/head:/src/ai/defaultai.cc - there are a lot of arbitrary numbers that all had to be tested, that means recompilation and running multiple games - it was pain and took a lot of time and therefore I decided to save the time spent on endless circles of edit/compile/test work and introduce the genetic algorithm as procedure for automatic self-tuning of constants.

Of course it has the disadvantages.

If you decide to commit to rewrite some parts of AI I will be helpful, with advice and explanation of my intentions in code and so. But it is really a commitment for years (for you). And dont afraid to rewrite parts of code to you liking as I did some time ago. Basically 90% of current AI code (src/ai/ folder) is mine, so I understand that mayor changes in logic might require rewriting from scratch...

[hessenfarmer (stephan-lutz)]

Hi
I have to admit that on a quick glance it looks not very comprehensible but after some thoughts I believe I understand what is going on in the code. While digging what preciousness is doing I probably found a bug giving preciousness a direct effect on the building necessity.
In
https://bazaar.launchpad.net/~widelands-dev/widelands/trunk/view/head:/src/ai/defaultai.cc#L5150

there should be probably inputs[i] instead of inputs[1].

[TiborB (tiborb95)]

EXACTLY!!!! good catch!

I can fix it next week do do some training so that things settle down after such significant change..

[ypopezios (ypopezios)]

I read the inputs[1] myself and I supposed that it is some kind of mathematical interpolation around the basis of the array (I forgot that arrays in C++ are 0-based), so I didn't bother looking at its value. My main issue is that such kind of bugs should be caught very early by printing some output and comparing it with the desired one. But current lack of structure makes that impossible, as it turns the code into a big "black box", meaning that such bugs can survive for many months, negating some improvement efforts during that time. Most tests should not need to run the game at all, they should instead be running specific functions against predefined inputs and comparing them with predefined outputs. That saves tones of time/pain and prevents most bugs from hiding for long times. It's a pity to spend so many hours coding and testing, just to have a few typing mistakes destroy the final result.

Any function that in readable fonts doesn't fit within the average computer screen is structureless code by definition, like a text without paragraphs. Individual statements can still be read and understood, but the bigger picture gets lost, so the bigger implications of small changes become a guesswork. Thus when I say about rewriting the code, it's not about turning it into my liking, but about applying basic coding principles, which are applicable to every computer language.

Having said that, it is acceptable to use structureless code during internal development and conducting tests on your own. But when the time comes to make the code public, you have to make it readable, not just working. The AI code should never be approved in its current form by the reviewers, as it doesn't conform to any principle. Therefore, let's combine Tibor's experience with my discipline and upgrade Widelands' AI.

[TiborB (tiborb95)]

I dont defend coding style, could had been done better of course.

But one of problems is that output of AI is too fuzy. Yes, some decisions can be clearly evaluated binary GOOD/BAD, but most decisions can be evaluated only very approximatelly (in range like in between 'very good' and 'absolutely bad')

This problem is visible also in training - all I was able to come up is a score that is mix of land size, count of soldiers, delta of land and soldiers and so on. Another problem is that we are not able to evaluate e.g. placing of single building. You as a human can "see" if a (e.g.) ranger was placed on good place on in right time, but there is no heuristic available to evaluate it automtically. Or at least I dont dare to create such heuristic because there is so many considerations to take into account. The same with so many aspects and decisions that AI does.

But I can imagine unittest for some smaller/partial functions.

So as conclusion - just go on...

[GunChleoc (gunchleoc)]

Migrated to https://github.com/widelands/widelands/issues/3275