Different http_etag for same person resource when accessed via people collection or team object

[dobey]

There seems to be cases where there is a different http_etag value for resources, when accessed in different ways. For example, in tarmac, we have a plug-in to check that authors in a branch are members of a set of teams. We use this in Ubuntu One code when landing branches, to automatically check that the contributors have signed the Canonical Contributor Ageement where appropriate, as we have a team which lists those people. However, for some people, it seems that the person resource when accessed via launchpad.people[username], has a different ETag when accessed via team membership listings.

For example, for one user, we get this: "7b0cf391e5dba48c9407566b7fc97061969ecf44-17ed7d0e7d9d613f4a30f54a7b029c63fb928db6" vs. "9d634e4c7e7c8bbc05c03fcc3b87c5c2d0bbe83e-17ed7d0e7d9d613f4a30f54a7b029c63fb928db6"

This causes are contributor check to fail, when it shouldn't. As you can see, even the second portion (after the -) of these hashes is the same for both. Why is this happening?

[Martin Pool (mbp)]

Could you please give the example URLs you're using?

Martin

[dobey (dobey)]

I am using launchpadlib, so I'm not sure what URLs you're asking for exactly. However, self_link is the same for both objects. Though http_etag is different for them. This causes the Entry.__eq__ to fail when it really shouldn't. And I've seen this happen with multiple users.

I wrote a quick script to help me debug this issue, but I am not sure if it belongs in lptools or not. Certainly it doesn't belong there as-is, but maybe it would be useful with a lot more hacking on it.

[dobey (dobey)]

This is probably the most relevant block of code, as it is where the failure occurs. person is the result of launchpad.people.getByEmail(), and team is a result of launchpad.people[teamname].

    def __is_in_team(self, person, team):
        """Check that a person is a member of team, or one of its subteams."""
        for subteam in team.members:
            if subteam == person:
                return True
            if subteam.is_team and self.__is_in_team(person, subteam):
                return True
        return False

[Martin Pool (mbp)]

I'm not sure if comparing for object equality is really the right way
to say "are these the same person?" I don't know what launchpadlib
documents as the semantics of equality, if anything. I would probably
check equality of self_link. This is not to say this bug is invalid
though.

[dobey (dobey)]

It is not launchpadlib, but lazr.restfulclient that is doing the comparison. And it's comparing self_link, http_etag, and an internal dirty dict it keeps around. The only differing point is the http_etag in that comparison. I am aware of the workarounds, but they are workarounds. If lazr.restfulclient is going to offer an __eq__ to compare the objects with, then it should work reliably. And if Launchpad is going to give me a person resource, for a specific user, that resource should be the same, no matter where it is pulled from. I don't know why I started seeing this last week, or why it hasn't happened to a lot more people, but I'm surprised it hasn't. And I'm not sure it is limited to the person resource. It is the only place I am currently seeing the problem, and so the only example I have to offer.

[Martin Pool (mbp)]

On 7 July 2011 06:51, Rodney Dawes <email address hidden> wrote:
> It is not launchpadlib, but lazr.restfulclient that is doing the
> comparison. And it's comparing self_link, http_etag, and an internal
> dirty dict it keeps around. The only differing point is the http_etag in
> that comparison. I am aware of the workarounds, but they are
> workarounds. If lazr.restfulclient is going to offer an __eq__ to
> compare the objects with, then it should work reliably. And if Launchpad
> is going to give me a person resource, for a specific user, that
> resource should be the same, no matter where it is pulled from.

I sympathize, but making sure there's no more than one local proxy
object for any remote object can be a hard problem. In general I
don't think lazr/lplib solves that problem perfectly; at least I have
seen other cases where they get out of sync. (For instance, iirc,
when objects are returned from a collection.) It's reasonable to me
that lplib would interpret eq as "objects are identical" rather than
"objects refer to the same remote object"; perhaps it's even more
reasonable. If that's true, then comparing the URLs is not a
workaround but in fact the correct client behaviour.

So I think to start with lplib needs to document how it promises this
should work.

[dobey (dobey)]

OK. Well if the etags shouldn't be compared, then lazr.restfulclient should probably only compare the self_link.

But, I still wonder why some of these entries are different at all. The people I'm hitting the failure case on aren't heavy launchpad users. And for comparing people that are heavy launchpad users (such that the objects may change frequently), this hasn't come up, at all. It's very weird.

[Martin Pool (mbp)]

There are two possible comparison functions:

1- "Are these two local proxy objects identical?" In this case it is
reasonable and probably necessary to compare the etags.

2- "Do these two proxy objects refer to the same remote object?" In
this case you should check the self_link.

lplib/lazr apparently does #1. I think that is reasonable.

This particular bug may not have been hit before but I've hit other
bugs coming from the same basic thing.

[dobey (dobey)]

I think lplib/lazr doing #1 for managing the cache is reasonable. I think doing it as part of the general API that developers end up hitting when writing scripts is probably not reasonable, since they aren't managing the cache. In the case of external API usage, I think #2 is more reasonable and what is expected. But given that lplib/lazr need to maintain a cache of proxy objects, they should be able to do #1 internally where needed. I just don't think that should be generally exposed through normal API usage.