GNUSim8085 as a regression testing framework?

[Onkar Shinde]

There was some discussion over at the sdcc forums about the lack of a simulator/emulator that could be used for regression testing purposes. Having a simulator/emulator that could be used to do regression tests would be very useful.
So the question is how complicated it would be to make use of the simulator in GNUSim8085 without necessarily relying on the GUI.
I am envisioning a different startup mode, where the GUI is replaced with a command line/console interface, so that we could run a given source file in a console environment, where the simulator would optionally respond with a dump of the internal state at an arbitrary point, possibly using a markup format such as XML or RDF.

Implementing this would probably be pretty straightforward if the simulator functionality can be easily decoupled from the GUI, i.e. if the simulator is following the MVC pattern.

In addition, this would really not be very complex at all - we would only need to accept a filename and one switch for a mode, i.e. step by step vs. complete runs.

And then, we'd only need to output all internal simulator state (mostly registers, flags, stack) to the console, so that it can be examined by utilities such as "expect".

So, how complicated would it be to have the simulator run in a very limited mode, WITHOUT the GUI?

I envision something along the lines of

./gnusim8085 test.asm

where test.asm would contain:
;---------
mvi a,05h
mvi b,03h
add b
;---------

and running the simulator would simply dump all relevant internal state to the console, preferably in some standardized markup/format.

http://sourceforge.net/forum/forum.php?thread_id=2030455&forum_id=1864

[Bug Importer (bug-importer)]

there's at least one other use for this, too: regression testing the GNUSim8085 source code!

[Bug Importer (bug-importer)]

yes, if there's a way to have the simulator dump internal state to the console, it would be pretty straightforward to use it for regression testing all supported instructions, so one could run very self-contained unit tests like:

lxi h,1024

and then dump all registers/flags, so that it could be verified if the h/l registers really contained the hi/lo bytes as expected.

[Bug Importer (bug-importer)]

there could be two different modes in "cli mode": "source input" and "command input".
That way, entering source code and commands can be easily separated.

Having a way to dump simulator state in some form of XML or RDF would be useful for processing it via other tools such as expect.

On the other hand, it would probably be easier to also provide an "expect directive" directly in the command mode, so that the simulator itself could directly tell if a test has failed or not

[Bug Importer (bug-importer)]

One could link to the readline library, which would simplify the whole thing.

[Abhay Buch (abhayb)]

Hi,

I just joined the group (thought what you guys are doing is really cool). I'm somewhat unfamiliar with Linux GUI development so I thought I'd start out working on something involving the CLI. Attached is a patch (run with -p1 in the gnusim8085 directory) that adds some basic command line functionality.

When called with "gnusim8085 -r <file_list>", it runs through each of the files, reads it as an assembly file, executes it and prints a dump of the registers and flags to stdout. Errors and messages go to stderr. The format of the dump is specified in primary file "cli-reg-test".

I plan to add much more (memory dump, varying levels of verbosity, expect directives and the like), I just thought I'd run this by everyone first.

I've tested it pretty thoroughly, but I'm still pretty unfamiliar with some of the code. If I've done something to break everything horribly, please let me know and I'll fix it immediately.

[Onkar Shinde (onkarshinde)]

Abhay,

Thanks for the patch. We are almost done for 1.3.7 version. I hope to release it next weekend. After that I will take a look at patch.

[Debjit Biswas (debjitbis08)]

I found this, 8080/8085 CPU Exerciser: http://www.idb.me.uk/sunhillow/8080exerciser/index.html
Is this of any help?

- Debjit

[The Escapist (wisd00m)]

I have not yet taken a look at the patch, but I would also recommend to look into using XML or RDF as the underlying markup language (i.e. not just plain text), that would ensure a high degree of consistency for the output, so that it can be clearly specified and standardized.
It would in fact be possible to come up with an XML schema for validating the created output.

Another neat thing is that an XML/RDF based format would be easy to provide backward compatibility for:

One important thing to do would be providing a "version" attribute for the created output, so that tools parsing the GNUSim8085 output, can always refer to this attribute for using the right DTD/schema.

If someone really NEEDS to have a plain text dump for some reason, it would be trivial to use XSLT to format the XML file differently, in fact it could even be automatically converted to HTML.

An XML/RDF based output format would be most detailed, so "reducing" it to less details will always be easier than enriching a flat plain text output.

Tools making use of this could in fact trivially run an XPATH expression against the created output, for example to only get a register dump.