bluetooth/detect shouldn't run on a system with no Bluetooth device

Bug #862322 reported by Nicola Heald on 2011-09-29
36
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Checkbox
Medium
Brendan Donegan
Nominated for Oneiric by Brendan Donegan
checkbox (Ubuntu)
Undecided
Unassigned
Oneiric
Undecided
Unassigned

Bug Description

Running the latest oneiric, checkbox and checkbox-gtk from ppa:checkbox-dev checkbox_0.12.8+bzr1082+201109290705~oneiric1 inside a Virtualbox virtual machine.

I get to the bluetooth tests, and get a dialog box saying "bluetooth/detect crashed" I have no bluetooth hardware attached.

I try to submit a bug report but the process freezes and I can't carry on with the test run.

[Impact]
The bluetooth/detect test will run (and fail) on systems with no Bluetooth hardware.
On systems with apport enabled, this will pop up a crash report window, needlessly worrying the user.
On all systems, this will mistakenly report to Ubuntu Friendly that bluetooth hardware was present but failed, which penalizes the user's score for optional hardware that is not actually present. This means that a computer without bluetooth hardware is unable to get a perfect 5-star score in UF, which is misrepresentative of a system that may work perfectly but have no Bluetooth support.

[Development fix]
Checkbox revision 1114 changes the test definition so it requires an actual bluetooth device to run. Otherwise the test is marked as skipped due to unmet requirements, which is the correct behavior.

[Stable fix]
This is fixed in http://bazaar.launchpad.net/~checkbox-dev/checkbox/trunk/revision/1114. Also, the linked branch (https://code.launchpad.net/~roadmr/ubuntu/oneiric/checkbox/0.12.8.1) contains this and other fixes and merges cleanly against Ubuntu checkbox 0.12.8.

[Test Case]
- Start "system testing" (checkbox-gtk version 0.12.8) on an 11.10 system with NO bluetooth hardware (check that output of hcitool dev shows no devices). As originally reported, a virtual machine may fulfill this requirement.
- Press "Next" and provide password when prompted.
- Press "Deselect all", then select "Bluetooth tests" and press Next.

Expected result:
- Bluetooth test doesn't run
- Test gets marked as skipped / requirements not met in the final report.

Actual result:
- Bluetooth test attempts to run, crashes, if apport is enabled the user will be requested to file a bug report.
- Test appears as failed in the final report.

[Regression potential]This actually reduces the pool of systems on which this test will run, and the added code simply checks that a bluetooth resource (essentially a string match against a text file) is present. Thus the potential for regression is almost nil. Maybe if a system has a bluetooth device that is not presented as such it will be untested now, but such a scenario is unlikely.

Related branches

Nicola Heald (notnownikki) wrote :

If i say that I don't want to submit a bug report, the run continues to the bluetooth file transfer test, even though I don't have the hardware.

Ara Pulido (ara) wrote :

Can you attach checkbox.log and submission.xml, please?

Both need to be under .cache/checkbox

Changed in checkbox:
status: New → Incomplete
Nicola Heald (notnownikki) wrote :

checkbox log from after the test is reported as crashing

Nicola Heald (notnownikki) wrote :

There is no submission.xml in .cache/checkbox.

There is no output from hcitool dev | tail -n+2 | awk '{print $2}';

Nicola Heald (notnownikki) wrote :

Just been asked to run:

hcitool dev | tail -n+2 | awk '{print $2}'; hcitool dev | tail -n+2 | awk '{print $2}';

No output from that command.

$? gives 0

I can't reproduce this - bluetooth/detect does *fail* on my VirtualBox, but that's to be somewhat expected. I'll take a look through the logs and see if I notice anything.

Mike, can you also attach subunit.log?

Nicola Heald (notnownikki) wrote :

Ok, so bluetooth/detect (and all Bluetooth tests) should really require: device.category == 'BLUETOOTH', but it doesn't. Given that the test only impacts on portable systems and the chances of finding a modern portable system *without* Bluetooth are slim to none, this isn't very urgent - but we should get the fix in for P anyway.

Changed in checkbox:
status: Incomplete → In Progress
importance: Undecided → Medium
assignee: nobody → Brendan Donegan (brendan-donegan)
milestone: none → 0.13
Ara Pulido (ara) wrote :

Let's fix it in the first Oneiric sru for checkbox

Changed in checkbox:
milestone: 0.13 → 0.12.9
Ara Pulido (ara) wrote :

Moving again to 0.13
Finally we are going to ignore errors in /detect

Changed in checkbox:
milestone: 0.12.9 → 0.13

I've stopped working on this one for now. It isn't urgent at the moment.

Changed in checkbox:
status: In Progress → Triaged
assignee: Brendan Donegan (brendan-donegan) → nobody
summary: - bluetooth/detect crashes
+ bluetooth/detect shouldn't run on a system with no Bluetooth device
Changed in checkbox:
status: Triaged → Fix Committed
assignee: nobody → Brendan Donegan (brendan-donegan)
milestone: 0.13 → 0.12.9
Daniel Manrique (roadmr) on 2011-11-28
Changed in checkbox:
status: Fix Committed → Fix Released
Daniel Manrique (roadmr) on 2011-12-02
description: updated
Daniel Manrique (roadmr) on 2011-12-05
description: updated
Martin Pitt (pitti) wrote :

Is this also fixed in precise's 0.13? If no, please fix it there first, otherwise please close the task. Thanks!

Daniel Manrique (roadmr) wrote :

As per Brendan's comment, and since the revision implementing this change is integrated into 0.13, I'll mark the Precise task Fix Released.

Apologies for the mixup, and thanks for reviewing these changes!

Changed in checkbox (Ubuntu Oneiric):
status: New → Fix Released
Daniel Manrique (roadmr) wrote :

Oh, I'm really messing things up today. I shouldn't have marked the Oneiric task Fix Released, not until the SRU has been published.

My sincerest apologies for this mess. I'll put the task back to New.

Changed in checkbox (Ubuntu Oneiric):
status: Fix Released → New
Chris Halse Rogers (raof) wrote :

Marking as fix released in Precise.

Changed in checkbox (Ubuntu):
status: New → Fix Released
Changed in checkbox (Ubuntu Oneiric):
status: New → Fix Committed
tags: added: verification-needed

Hello Mike, or anyone else affected,

Accepted checkbox into oneiric-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Daniel Manrique (roadmr) wrote :

OK, I ran the test case with both 0.12.8 and 0.12.9 and in both cases on a system with no Bluetooth, the tests are marked as failed, meaning that checkbox is trying to run them even if no bluetooth hardware is present.

The cause for this is that 0.12.9 doesn't contain the required code fix, even though the changelog says it does.

This was an oversight on my part. Apologies for this. I'm linking a branch that can be merged against lp:ubuntu/oneiric/checkbox to fix this by adding the necessary bits of code.

https://code.launchpad.net/~roadmr/ubuntu/oneiric/checkbox/fix-662322-in-0.12.9

I'm marking this verification-failed and await instructions on how to proceed.

Thanks!

tags: added: verification-failed
removed: verification-needed
Martin Pitt (pitti) wrote :

Hello Mike, or anyone else affected,

Accepted checkbox into oneiric-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

tags: removed: verification-failed
tags: added: verification-needed
Daniel Manrique (roadmr) wrote :

Hi,

I ran this on a Dell Inspiron 1545 with no Bluetooth adapter.

I deselected all tests in checkbox, and reselected only the bluetooth category.

Checkbox 0.12.8 marks the bluetooth/detect test as FAILED (old, wrong behavior).

I then upgraded checkbox and checkbox-gtk to 0.12.10 and reran the bluetooth tests. This time they appear in grey in the report, saying "Job requirements not met", meaning the jobs were skipped altogether and don't count as a failure, which is the desired behavior.

I'm adding verification-done to this bug.

Thanks!

tags: added: verification-done
removed: verification-needed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package checkbox - 0.12.10

---------------
checkbox (0.12.10) oneiric-proposed; urgency=low

   [Brendan Donegan]
   * Have the bluetooth/detect tests require a device with the category
     BLUETOOTH to run, thus preventing the test from failing on systems with
     no Bluetooth device (LP: #862322)
   * Remove fix for bug 877752 on connect_wireless, as it was not working as
     intended.
 -- Daniel Manrique <email address hidden> Tue, 07 Feb 2012 15:30:36 -0500

Changed in checkbox (Ubuntu Oneiric):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers