HTML elements without id

[Leo Arias]

All HTML elements that will be part of the automated tests (i.e., all of them :) must have an unique and meaningful id.

On the login page ( go to https://login.ubuntu.com/ without a session started ), the elements without an identifier are:
- The button "Continue"
- The link "Find out more"
- The button "New Account"

On the decide page ( go to https://staging.one.ubuntu.com/auth/login , fill your credentials and press continue ), the elements without an identifier are:
- The button "Yes, sign me in"
- The link "Cancel"

[Michael Nelson (michael.nelson)]

Hi Leo! SSO is not actually a project I work on except when they're short, but I am nonetheless interested to know why you need ids on all the elements for testing? We wrote a bunch of simple-selenium tests for SSO and enjoyed the fact that we *didn't* need to go updating the html with ids just to support functional tests (as once you go adding ids, the next dev to make some style changes invariably uses them for style rules etc.)

[Leo Arias (elopio)]

Hi Michal,

Good ids are important to make tests readable and easy to maintain. But the most important thing is that they are the only way to be sure that you are using the element you are supposed to. Many elements can match whatever xpath expression you use to locate things; and even if you make it specific enough to match only one element, other elements will be added to the page and they might make the test fail. Or even worst, the test can pass but will not be doing what you meant when you first programmed it. This happens on the u1 services page, for example, where two buttons without id can not be easily distinguishable, and we have to use an index that might change in the future. For me, that's a testability bug.
So, elements without ids make UI tests even more fragile than they already are. In my opinion, adding ids to elements should be a development rule, and not using them for css styling should also be one :)

Where do you have those tests you made? I bet we have duplicated a little of your work :(

pura vida.

[Michael Nelson (michael.nelson)]

On Thu, Sep 8, 2011 at 7:29 PM, Leo Arias <email address hidden> wrote:
> Hi Michal,
>
> Good ids are important to make tests readable and easy to maintain. But the most
> important thing is that they are the only way to be sure that you are using the
> element you are supposed to. Many elements can match whatever xpath expression
> you use to locate things; and even if you make it specific enough to match only one
> element, other elements will be added to the page and they might make the test
> fail. Or even worst, the test can pass but will not be doing what you meant when
> you first programmed it. This happens on the u1 services page, for example, where
> two buttons without id can not be easily distinguishable, and we have to use an
> index that might change in the future. For me, that's a testability bug.

I think I would agree - if there wasn't a way to distinguish the
buttons without ids, but I don't understand why there isn't a way
without using ids... such as using the name attribute [1]:

button_click(get_element(name='continue'))

or if that's not unique, even the text on the button [2]:

confirm_button = get_element(tag='button', text='Yes, remove')

Of course, technically, you could have a page that has multiple
buttons with "Yes, remove", but I think that would be (1) a serious UI
issue, and (2) an exception to the norm. Anyway, I'd assume they'd
still need to have different name attributes.

> So, elements without ids make UI tests even more fragile than they already are.

If you're relying on the order of the buttons, then yes, but I'd hope
there was another way as above, that relies on a sensible user
interface (ie. only one button with name="do_foo" and the text "Yes,
do foo").

> In my opinion, adding ids to elements should be a development rule,

hrm, I don't see why, as above. I think this is only necessary if
you've got (1) a UI doing very strange things, or (2) a test framework
that doesn't let you select easily (ie. using css selectors or
similar).

> and not using them for css styling should also be one :)
>
> Where do you have those tests you made? I bet we have duplicated a
> little of your work :(

https://code.launchpad.net/isd-sst-tests

Let me know if you don't have access and I'll make sure you're
subscribed to the branch.

[1] https://bazaar.launchpad.net/~canonical-isd-qa/isd-sst-tests/trunk/view/head:/tests/shared/sso_util.py#L36
[2] https://bazaar.launchpad.net/~canonical-isd-qa/isd-sst-tests/trunk/view/head:/tests/shared/sso_util.py#L52

[Leo Arias (elopio)]

In your first example, you are using the name as an identifier. It's not bad and it works, but it can be better. Instead of telling the website developers: "You should make sure that the combination of tag, text, class and name uniquely identifies your elements", you could tell them: "Set a unique id for the elements that should be tested".

The id is the only element that's required to be unique by the specification [1]. So, having two elements with the same name is not a bug, and there are lots of cases where having the same text and class is required. In your second example, you add the text, but that doesn't always work, and you can see the Services page example I mentioned. This page has no visible UI issues, it makes perfect sense to have the same text on both buttons. The bug there is not the lack of a unique name/text/class/... combination, it's the lack of an id.

We always have a workaround making xpath expressions as complex as required, so we might think of this as a low priority issue; but as it has the chance to break the tests so easily, I consider this as a major problem. Expanding on what I mentioned before, button_click(get_element(name='continue')) will work until somebody adds a second button named continue. The fault will not be of the developer, it will be the tester's fault for relying on something that doesn't uniquely identify the element. get_element(tag='button', text='Yes, remove') will work until somebody decides that the text should be shortened to 'Yes'. The fault here will be of the tester for relying on something that's likely to change.

Another important point is that elements like span [2] don't have a name attribute. For consistency and ease, all elements must be uniquely identified by their id. It makes a lot of sense to me :)

Finally, for readability and maintainability while using sst, this is much better: button_click('continue')

This, of course, is just my opinion. I think that source code must be written with testability in mind, and that by trying to make the tests for your code as easy as possible, your code will be of a higher quality. As a result, your tests will also be of a higher quality. But we can make some compromises here, like setting the importance of this bugs to medium :D

pura vida.

[1] http://www.w3schools.com/tags/att_standard_id.asp
[2] http://www.w3schools.com/TAGS/tag_span.asp

[Leo Arias (elopio)]

Oh, I forgot. I don't have access to your branch. Please let me read it.
We'll need to find some way to use your tests instead of rewrite them.

[Michael Nelson (michael.nelson)]

On Fri, Sep 9, 2011 at 4:58 PM, Leo Arias <email address hidden> wrote:
> In your first example, you are using the name as an identifier. It's not
> bad and it works, but it can be better. Instead of telling the website
> developers: "You should make sure that the combination of tag, text,
> class and name uniquely identifies your elements", you could tell them:
> "Set a unique id for the elements that should be tested".

...except that we'd then be adding quite a lot of markup to the page
that exists only for testing, and that also tempts css writers to use
those ids. I think it comes down to the fact that we're testing
something that does change, and occasionally tests will break because
the app has changed (that's a good thing), it's just deciding on a
balance of how often you get false positives - I think there are pros
and cons both ways. My rule of thumb is to use meaningful/semantic CSS
selectors for functional tests - using the existing markup - where
possible.

> So, having two elements with the same name is not a
> bug, and there are lots of cases where having the same text and class is
> required.

Sure there are cases like that - and in those cases I'd add an id if I
can't make a non-fragile test without it. It's just the idea that
"adding ids to elements should be a development rule" that's strange
for me... but as said, it's not my project, I was just interested in
the discussion :)

> In your second example, you add the text, but that doesn't
> always work,

That is true - it doesn't always work, but it wasn't an example to
show a rule that always works... Maybe that's what you're after, in
which case I'd say, yes the only way to do so is to add an id - that
will enable you to access all elements in the same way (via a unique
id). I'd not recommend doing that myself, but that's not relevant here
:)

> and you can see the Services page example I mentioned. This
> page has no visible UI issues, it makes perfect sense to have the same
> text on both buttons. The bug there is not the lack of a unique
> name/text/class/... combination, it's the lack of an id.

That may be the case for that example - but I still don't want to add
ids to every element just because there are situations where I need
ids.

>
> We always have a workaround making xpath expressions as complex as
> required,

I think css selectors are much easier to read, if they're available to you.

> I think that source code must be
> written with testability in mind, and that by trying to make the tests
> for your code as easy as possible, your code will be of a higher
> quality.

I agree but don't think that means adding ids to every element. Adding
an id where one is required because the test is too fragile or unclear
- great - adding an id to every element required for testing... hrm,
that's where I'm not convinced (yet) :) That said, I'm not a QA person
- so I might just be thinking like a dev - perhaps it's obvious to
anyone who's written lots of functional tests...

> As a result, your tests will also be of a higher quality. But
> we can make some compromises here, like setting the importance of this
> bugs to medium :D

heh :)

> Oh, I forgot. I don't have acce...

[Leo Arias (elopio)]

Hey guys.

We have discussed about this on the u1 team, and reached a solution that made the developers and QAs happy.

You might like to take a look: https://blueprints.launchpad.net/ubuntuone-testing/+spec/identifiers-for-webui-testing

[Michael Nelson (michael.nelson)]

On Wed, Sep 28, 2011 at 6:38 PM, Leo Arias <email address hidden> wrote:
> Hey guys.
>
> We have discussed about this on the u1 team, and reached a solution that
> made the developers and QAs happy.
>
> You might like to take a look: https://blueprints.launchpad.net
> /ubuntuone-testing/+spec/identifiers-for-webui-testing

Cool - thanks Leo, I just had a browse through and added the following
question to the discussion page:

Hi Leo, and thanks for all the work that's gone into this. The only
point that is unclear for me from above is whether you think it best
to add the data-qa-id to every element that your QA tests will use, or
just when a meaningful identifier matching the user experience is not
possible? Second point regarding the 'Con' of longer sst statements,
if the sst guys think the data-qa-id a great idea, I'm sure they won't
mind updating button_click and friends to check for data-qa-id after
id (or similar) -- Michael Nelson 2011-09-29 08:02:33

[Leo Arias (elopio)]

I'll answer in the blueprint, to keep the discussion in order and to let Stuart and Rick join with their thoughts.

pura vida.