Juddering and disappearing road traffic when loading shapes and textures

[Mervyn Burse]

I have noticed that I am getting the warning that some of the STF file was not parsed, and at the same time I get a lot of screen judder, similar to the MSTS file load judder.

The warnings show which world file is not fully parsed and the line number involved. Without exception the line number is the EOF closing bracket with the following line having ERROR: **** Unknown Message-Definition ('Msg100','en') ****

When this happens there is also a 50% chance that road traffic disappears as well. Not the single vehicle hitting the end of the spawner track, but all traffic in view in both directions.

I have looked at the Wfile.cs and see that there is code there to stop this from happening, but it does not pick up lines like that shown above.

Mervyn

[James Ross (twpol)]

The warning reported is correct - and useful if you accidentally have an extra ")" in your file - so I would rather not work around that. Code that currently exists to skip to the end of the file is bogus IMHO.

We are going to have to work out why the road traffic disappears - that should not happen. Likewise, the stuttering should not be happening, though it might be a consequence of outputting log messages, which could make it a tricky problem to solve at this time.

Please try running with logging disabled and see if that makes any different to either of these issues.

[Eric Swenson (butter-cup)]

The following message is appended to the world files of the Pahausett Electric route (freeware at Trainsim.com).

_Skip ( "TS DB-Utility - Version: 3.4.08(10.06.2010), Filetype='World', Copyright (C) 2003-2010 by ...CarlosHR..." )

Apparently TSUtils adds this when it is run to check a route.

[Mervyn Burse (mervyn)]

James

I will turn logging off and see what happens.

Eric

There is code in Wfiles.cs that handles the TSutils world file addition. It is not picking up the other thing that I have found in the World files that have the problem.

[James Ross (twpol)]

Eric, the parser should be coping with _SKIP and COMMENT and other ignored blocks in the file - including when it decides whether to show the 'was not parsed' warning. If it isn't, that's a bug we need to fix too.

[Eric Swenson (butter-cup)]

Unless there is something else in the world files it's not picking up the _skip. I'll have to open more w files and check to see if they all have the appended line. Is there a way to easily detect the last closing ) and then not read beyond that?

[James Ross (twpol)]

I'm not sure what you're asking. OR will read up to the end of the expected data, i.e. the last closing ), and then warn if that isn't the end of the file. When checking if it is at the end, it should ignore any _skip() and comment() blocks (along with anything else it would normally ignore).

[Eric Swenson (butter-cup)]

I was thinking about how to eliminate the warning if there was data outside the normal file boundary.

[Eric Swenson (butter-cup)]

Just finished using Route Riter to compress/decompress the world files of the Pahausett Electric route. This processing removed the appended _skip .... . OR warning no longer appears in the log.

[Mervyn Burse (mervyn)]

James

Have run the same route/activity without logging enabled. The screen judders and disappearing road traffic still occur. Therefore it has nothing to do with producing the log file. The judder and traffic going AWOL happens when the Load Process goes over 65-70%. The road traffic actually freezes and then gets wiped from the scene.

I have the route on an SSD, OR on an HDD.

Mervyn

[Mervyn Burse (mervyn)]

Further information. I am now satisfied that it is not the failed parsing, although all those messages flying up the screen at the time gave that impression, that is the cause of the judders and road traffic problems.

I had the logfile window where I could see it along with the onscreen debug info. It is the shape/texture loading that is the root cause, exactly the same as MSTS. No problem where there are only one or two shapes involved, but when the required numbers go into double figures, the judders increase, and eventually overload the load process which causes the road traffic to freeze and disappear.

[Mervyn Burse (mervyn)]

Additional info after further testing on both OR and MSTS with identical equipment.

The screen judders when loading shapes and textures is far worse in OR than MSTS at the same points on the route. MSTS retains the road traffic, OR drops it after the screen freezes momentarily.

This happens exactly the same whether I use 1500m viewing distance ( 9 tiles ) or 2500m ( 25 tiles ). The shape and texture load is less with the smaller tile load, but the nasty judders etc do not change.

I notice that with the X1622 there are no more world file parsing errors.

[Mervyn Burse (mervyn)]

World File parsing error is back with X1625.

With regard to the disappearing road traffic, I have also noticed that when the load process % starts to hit the higher numbers, the CPU % starts to go down. The Normal CPU load for the Surfliner 2 route is between 55 and 60%. When the load process stream starts to reach into the higher figures, the cpu load drops to about 35%, then at the point where the screen freezes momentarily and traffic disappears, the cpu load shoots up 104%, then drops back to previous normal levels.

I am running a Core2 Duo E7500 clocked at 2.93Ghz along with 8Gb of DDR2 with Win7 x64

[James Ross (twpol)]

Thank you for the detailed investigation work Mervyn; I am convinced there is a serious issue here that needs fixing. I am 'converting' this bug in to one about the juddering and road traffic disappearing (since they seem related) and not about the STF parsing. That might need investigating too, but it doesn't seem to be a serious issue.

[James Ross (twpol)]

Mervyn, could you post a screenshot of the HUD/F5 DEBUG information and a brief summary of your computer specs, please?

[Mervyn Burse (mervyn)]

I have made a set of shots taken at different stages of the process and a shot of my DXDIAG info.

[James Ross (twpol)]

Thanks.

I haven't been able to get the same issues as you, but I have a couple of suggestions. However, as I am currently away from my PC for a day or two, I'll have to describe the code changes you could try. I hope you can try them out, but if you have any problems I'll make some pre-built versions tomorrow.

Change 1:

In Source\RunActivity\Processes\UpdaterProcess.cs, around line 36, where it says:
    Threaded = System.Environment.ProcessorCount > 1;
Replace that with:
    Threaded = false;

Change 2:

In Source\RunActivity\Viewer3D\Materials.cs, around line 58, where it says:
    Textures.Add(path, texture);
Add the following after it:
    Thread.Sleep(100);

And in Source\RunActivity\Viewer3D\Shapes.cs, around line 69, where it says:
    Shapes.Add(path, new SharedShape(Viewer, path));
Add the following after it:
    Thread.Sleep(100);

Please try these two changes separately, and feel free to experiment with the numbers in the two Thread.Sleep calls. Higher might work better for the stuttering but worse for loading speed.

Please let me know if either of these have any effect on the way Open Rails runs on your machine.

[Mervyn Burse (mervyn)]

Unfortunately, for some reason I cannot compile the code. I get The project file cannot be loaded for Launcher and Runactivity. Both are flagged with a red marker.

[James Ross (twpol)]

Seems likely that you haven't got the XNA Game Studio installed (http://www.microsoft.com/en-us/download/details.aspx?id=39) but if that's not it, wait until this evening and I'll have made some builds for you.

[Mervyn Burse (mervyn)]

James,

OK. Got the XNA sorted out and compiling now works. Change 1 did not seem to
make much difference. Change 2 however is quite startling in comparison.
Game load time is definitely increased. Ingame screen judders now occur much
more, but road traffic no longer disappears. I did however have one instance
in my short run where everything except the player train and the terrtex
vanished.

In most places the Loader process stays at 0 or 1 percent. In the places
where road traffic used to disappear, that goes up to between 60 and 90%. At
one stage after the world vanished it hit 100% with a 1% queue. In all cases
though, processor load did not exceed 65%. Going to experiment with the
figures and see where that takes me.

Thanks
Mervyn

-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2012.0.2242 / Virus Database: 3184/5888 - Release Date: 06/05/13

[James Ross (twpol)]

Change 2 made the juddering *worse*? That's somewhat unexpected, though if we have let something get blocked by the loader process (which should never be the case), that would explain it. I will have to investigate more tonight.

[James Ross (twpol)]

Please check out X.1633 and see if it makes any difference. I found a bug introduced in X.1422 which caused ALL the world files (including all scenery and things like road cars) to be reloaded from scratch on every tile boundary crossing.

[Mervyn Burse (mervyn)]

James

That made a difference. Load process does not run as high now, road traffic
remains visible, albeit a bit jerky. that is because there is still a bit of
screen stutter at those points. Maybe time to upgrade this machine again.

Thanks
Mervyn

-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2012.0.2242 / Virus Database: 3184/5889 - Release Date: 06/06/13

[James Ross (twpol)]

That's good news, at least; I'll keep this bug open for a bit and continue thinking and poking about the code in case anything else comes of it.

[James Ross (twpol)]

In X.1682 I've added an experimental option 'loading delay' which people can try to see if it helps with loading stutter. I recommend not increasing it by more than 100ms at a time, as it can have a significant effect on loading time.

[James Ross (twpol)]

I'm marking this as fixed because the main issue was addressed, but there is always more we can do - I think new bugs should be filed about specific cases where we perform badly.