Restructuring code of ubuntu-desktop-testing

[Markus Korn]

There is a thread on the ML about using buildout [0]. One thing discussed there is the reorganization of the code within the ubuntu-desktop-testing project. I've created a branch to implement the suggestion I made on the ML thread.
I'm not sure if the structure I choose is perfect, but at least to me it makes sense, and code is always better than thousand words ;)
So, what do you think about it? comments are welcome.

Markus

[0] http://mail.gnome.org/archives/desktop-testing-list/2009-April/msg00004.html

[Ara Pulido (ara)]

Hello Markus,

Thanks for your efforts!

I generally like the new structure, but I have some comments to make:

* Why do we have two folders for ubuntu-desktop-testing script?
  bin/ubuntu-desktop-testing
  cmd/cmd.py

  Is it completely necessary for buildout? I would prefer to have it under bin

* desktoptesting_testsuites:
I think this folder name is a bit long. I would prefer testsuites. If we are afraid that this can be confused with the desktoptesting tests themselves, we just can add an explanation on folder structure on the HACKING file. Mmm, now that I check the setup.py file, there, you call this folder just "testsuites" so I guess it was a late minute change ;-)

[Markus Korn (thekorn)]

This bin/u-d-t and cmd/cmd.py was my first approach to split this up a bit. In rev 71 of this branch I've moved all the TestSuitRunner related parts of the code to 'src/desktoptesting/testing_framework/', so I'm fine with putting all the script logic back to bin/
My concern is that a 'testsuites' modul is too generic, there is no indicator in the name to show that this module belongs to desktoptesting. So another possible solution would be to move this package as a subpackage into desktoptesting, so it becomes 'desktoptesting.testsuites', or maybe 'desktoptesting.core_testsuites'

[Javier Collado (javier.collado)]

Hello,

Markus, I second the thanks for you efforts. I must say that I also like the structure since in my opinion it's an improvement over what we have right now.

There are some minor things that I would have done in a different way, but in general I think that all of them are fine since I cannot think about something better right now. Anyway, I enumerate them below, just in case someone has any idea about how to make them even better:

- utils: It's a little weird to have a utils package and a utils module in the testing_framework package. However, I understand that the utils package is intended to help test suite writing and utils module in the testing_framework package is expected to be used only by the test runner so it's unlikely that both of them are used in the same script.

- ldtp_abstraction: I would have called this package just 'application' because it's shorter. However, I see that 'ldtp_abstraction' is a good name because it provides good information about what the modules under the package try to accomplish. An alternative name could be, for instance, 'wrappers'; but I don't like it much either

- desktoptesting_testsuites: I agree on that the name is a little bit long. Maybe it can be shortened to 'desktop_testsuites' or even 'desktop_tests'. I see in 'setup.py' that they are to be installed under 'share/ubuntu-desktop-tests/' so I believe that using the same name would be nice. Given that gnome tests are also included I could be a good idea to change 'share/ubuntu-desktop-tests/' to 'share/desktop-tests/' as well.

In addition to this, probably the __init__.py file in this directory isn't needed since I believe that the testsuites aren't expected to be imported but discovered by the test runner so they aren't designed to be a package. It's not wrong to have the __init__.py file, but I think it's not really needed.

Best regards,
    Javier

[Markus Korn (thekorn)]

Thanks Ara and Javier for your comments,
I understand that the naming is not perfect, I see your points and I will think about it again.

src/desktoptesting_testsuites/__init__.py is needed for a little trick I used to get rid of `TESTS_SHARE = "."`. Using the cwd here is problematic. For example when you try to run the ubuntu-desktop-tool from outside the package root no testcases will be found. I solved this issue by converting src/desktop_testsuites into a package, and using pkg_resources.resource_filename("desktoptesting_testsuites") to get the path to the testsuites.

Markus

[Jason Cozens (jason-cozens)]

Hello,

In the name "testing_framework" I think "testing" is redundant as this is the framework for desktoptesting so 'desktoptesting.framework' would be sufficient for the package. This would be in line with using 'desktoptesting.testsuites' and more consistent (?).

The issue with the utils package and utils module may disappear as I think many of the functions in utils.py should possibly be on the base class TestRunner or incorporated into another more specific class that can be included through inheritance.

Jason.

[Markus Korn (thekorn)]

Jason, I agree with you, "testing" in "testing_framework" is redundant, just "desktoptesting.framework" makes sense.

let's talk about the "testsuites" again. I'm new to this project, so maybe my understanding is not correct or I'm missing some ideas: There are two possible places for such testsuites, one somewhere in the ubuntu-desktop-testing project and one in "~/.ubuntu-desktop-test". This places are searched for testcases and matching testcases are loaded, and run. So for me there are basically two types of testcases: "core-testcases" (placed in u-d-t) and "non-core-testcases" (placed in $HOME)
The question which comes in my mind here is: why not making the "core-testcases" a project for its own, this would have several advantages, like you can give testcase contributors commit access only to "core-testcases" and not to the framework itself, or if a user only wants to test an application which has not testcase in "core-testcases" he/she can install the framework without the testcases and places the related testcases in $HOME

I somehow have the feeling that the testcases itself do not fit into the desktoptesting package.

But I might be completely wrong and missed some important points,
Markus

[Jason Cozens (jason-cozens)]

Markus, like you I'm new to this project.

My understanding is that at the moment the tests are in the project directory and the results of running tests by default were saved in ~/.ubuntu-desktop-testing. Your suggestion would move the tests into desktoptesting.testsuites.

I think at the moment it would make sense to leave the tests in desktoptesting.testsuites until the project has developed further. The location of the tests can be defined later with the possibility of a project for desktoptesting and a project for the tests themselves.

Cheers,
Jason

[Ara Pulido (ara)]

Markus, can you do this (based on the comments and your initial idea) once the rest of the merging tasks are done? Thanks.