| 2013-12-19 10:12:11 |
Dan Chapman |
description |
As an example bug 1261745, on jenkins it displays the error as the page title hasn't changed, meaning we didn't get to the next step of the install but actually the problem was we were stuck at an error dialog displaying the installer couldn't create a JFS partition.
This type of problem can also be seen on the lubuntu tests where the test result is reporting that the progress bar didn't reset but is in fact halted by an error dialog. So these results provide no useful information before further investigation as to what the actual cause for test failure was.
The key problem is the tests check for error dialogs at anticipated points throughout the install usually providing input and switching to the next step or while polling on the progress bar, which in reality the dialogs could pop up at any point of the install.
It would be a better idea to only check for visible error dialogs at the end of the test whether its a fail or pass, and try and grab as much info from the dialog as possible and add it to the test result, this way we will always get the info of any error dialogs open at the end of a test, and not need to try and guess where to catch them during. |
As an example bug 1261745, on jenkins it displays the error as the page title hasn't changed, meaning we didn't get to the next step of the install but actually the problem was we were stuck at an error dialog displaying the installer couldn't create a JFS partition.
This type of problem can also be seen on the lubuntu tests where the test result is reporting that the progress bar didn't reset but is in fact halted by an error dialog. So these results provide no useful information before further investigation as to what the actual cause for test failure was.
The key problem is the tests check for error dialogs at anticipated points throughout the install usually between providing input and switching to the next step or while polling on the progress bar, which in reality the dialogs could pop up at any point of the install.
It would be a better idea to only check for visible error dialogs at the end of the test whether its a fail or pass, and try and grab as much info from the dialog as possible and add it to the test result, this way we will always get the info of any error dialogs open at the end of a test, and not need to try and guess where to catch them during. |
|
