Add automated tests to prevent API breakage for Ubuntu.Web

[dobey]

The webbrowser-app source package is what provides the Ubuntu.Web QML component. However, this component is not properly versioned in a stable way that aligns with the UITK versioning. As a result, apps are broken simply by newer versions of the package being included in updates.

For example, if one has an application that uses Ubuntu.Web, but was still using the older Ubuntu.Components import, the newer version of Ubuntu.Web imports the newer Ubuntu.Components, which creates conflicts, and results in various aspects of the app being broken, in particular many elements have improper theme rendering.

As a supported part of the SDK, this should never happen, and QML imports should be version stable.

[Olivier Tilloy (osomon)]

Do you mean that the version of Ubuntu.Web should have been bumped when all the imports were updated to Ubuntu.Components 1.3¹ ?
If so, I agree but it seems quite late to do that change. The imports were bumped 7 months ago, and at the time I was not aware that UITK versions were incompatible with each other (IIRC this was discovered by accident, SDK folks didn’t anticipate it either), which is the reason why the version of Ubuntu.Web was not bumped.

It is definitely something that will be taken into account next time such a change happens. Not sure we want to do anything about it for the current version though.

¹ https://bazaar.launchpad.net/~phablet-team/webbrowser-app/trunk/revision/1120

[dobey (dobey)]

Well, ideally without breaking the older version imports, yes. Sure, not anticipating changes is only part of the problem though.

Like the ABI requirements we have for C/C++ libraries, we also need to have similar checks and requirements for the APIs we provide as QML plug-ins. Unfortunately, I don't know exactly how to do that. A good start at least, would be to have some automated tests in various places that fail when these issues happen, so that we can at least catch them before all the apps in the store get broken and we don't misplace blame onto the apps for "using the UITK wrongly," when the components are what broke, and not the apps.

[Olivier Tilloy (osomon)]

I agree, some automated tests to catch potential problems similar to that one upfront would be very useful. Let’s keep this bug open to track the addition of such tests, although I’m not sure what those tests would look like yet.