When recording a new test, if I edit stdout, my changes get lost

[Emily Bache]

I create a new test, I record a new use case, and while doing so notice a bug. I quit the application and view the test files that have been created. The bug has caused something wrong to be written to stout. I edit the stdout recording to indicate the bug. When I click "save" everything looks ok and green, and my change is still there. When I quit the dynamic gui and look at my test in the static gui, suddenly my edit has been removed. This is very confusing and will cause the test to pass even though I saw a bug and thought I had created a failing test for it.

If I instead (BEFORE I start recording the use case) select in that annoying dialog that I want to replay using the dynamic gui, then I can see explicitly when my edit is removed in the replay step. This helps me understand what happened but doesn't improve matters much.

What I want is to improve the usability around recording a new use case. Perhaps the annoying dialog should stick around while I'm recording the test, and I should be able to click something on it that says "I found a bug, use dynamic gui to replay, and I'll want to edit stdout later".

At present the only way to get around it is to note the bug (on a piece of paper so I dont forget), save whatever the test gives you, close the dynamic gui, then in the static gui to edit the saved stdout file, (provided I havn't lost my piece of paper in the meantime). This is a strange workflow, where I first save a "gold standard" copy of stdout that I know is wrong, then edit it later on.

[Geoff Bache (geoff.bache)]

For SWT (and also for Swing) we've taken the view that the output should not be generated at all for recording, partly because it's impossible to make it the same, and partly to avoid the temptation to edit it. Perhaps one solution would be to simply make sure there is nothing to edit until the autoreplay is complete.

The fact is, generated output when recording is only useful as a "vague preview", and editing it cannot be made to be useful. It has to be overwritten, whatever happens, because we are interested in saving the output from the replayer.

[Geoff Bache (geoff.bache)]

See above comment. I don't really know what to do with this, the only thing to do is to somehow make it more obvious that the output there cannot be edited. A possible way forward would simply be to remove it, as it seems unlikely newer toolkits will generate it at all.