removable_storage_test fails on server for USB devices

[Daniel Manrique]

[IMPACT]

 * checkbox job "usb/storage-preinserted" failed to run on server installations
   due to the requirement of needing root privileges to mount removable usb devices.

 * backport will ensure that automated storage-preinserted test within checkbox will
   run and not fail immediately due to the necessity of mounting a device as user (root).

[TESTCASE]

 * Do a server installation, then reboot with the USB stick plugged in.
 * Install checkbox-certification-server
 * Emulate the usb/storage-preinserted test by running this manually:

   checkbox-cli -W <(cat <<EOF
   `head -n 14 /usr/share/checkbox/data/whitelists/default.whitelist`
    __usb__
    usb/insert
    usb/storage-automated
    EOF
    )

   supply root password when prompted.

 * Expected result:
   - Success exit code and something like this in the log (this was obtained
     on a DESKTOP system):
     Found the following mounted ['usb'] partitions:
     /dev/sdb1 : /media/MUSHKIN
     --------------------
     Running ['usb'] file transfer test for 1 iterations
     Creating Temp Data file
     File name is :/tmp/tmpkCn07B
     File size is 1048576 bytes
     Parent hash is: 2ebe1e55057a5b0aadecbebfcb14077d
     Copying /tmp/tmpkCn07B to /media/MUSHKIN
     Hashing copy on /media/MUSHKIN
     Hash of /media/MUSHKIN/tmpkCn07B on /media/MUSHKIN
     is 2ebe1e55057a5b0aadecbebfcb14077d
     Successfully completed 1 ['usb'] file transfer test iterations

 * Actual result:
   - Error exit code
   mount: only root can do that
   Error: can't mount /dev/sdb1
   There're total 1 device(s) failed at mounting.
   No device being mounted successfully for testing, aborting

[Regression Potential]

 * This is a simple change to the usb/storage-preinserted and
   power-management/fwts_wakealarmjob within so that these test can run with
   the proper permissions.

 * We are changing a test that used to run without
   root privileges to now having them.

[Daniel Manrique (roadmr)]

As an update, this mainly affects usb/storage-preinserted, which is perhaps the only USB test that runs on servers, as per this failure:

https://certification.canonical.com/hardware/201204-10880/submission/N7AU24y3iSAQf2G/result/test:usb%252Fstorage-
preinserted:certification

So making that particular job definition user: root has less of an impact than initially thought. I'll propose that as a solution.

[Clint Byrum (clint-fewbar)]

Hello Daniel, or anyone else affected,

Accepted checkbox into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/checkbox/0.13.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Clint Byrum (clint-fewbar)]

Does this bug need to remain private? I'd rather not mention a private bug # in an SRU changelog unless its really critical that it remain private.

[Daniel Manrique (roadmr)]

Hi Clint,

No need for it to be private, it was just an oversight which I've just corrected. Thanks!

[Daniel Manrique (roadmr)]

There is a small problem with the test command. The rest of the description mentions usb/storage-preinserted, but the test command says usb/storage-automated. I'll try to verify with this oversight corrected.

For reference, here's the correct test command:

checkbox-cli -W <(cat <<EOF
 `head -n 14 /usr/share/checkbox/data/whitelists/default.whitelist`
    __usb__
    usb/insert
    usb/storage-preinserted
    EOF
    )

[Daniel Manrique (roadmr)]

OK, so testing this, with checkbox 0.13.7 installed, I ran the test command from the above command. It failed with something relating to udisks not being present. It seems that the usb tests require udisks, so I installed, then retested.

I got a failure and this blurb in the submission.xml file:

<question name="usb/storage-preinserted">
<answer type="multiple_choice">fail</answer>
<answer_choices>
<value type="str">fail</value>
<value type="str">pass</value>
<value type="str">uninitiated</value>
<value type="str">unresolved</value>
<value type="str">unsupported</value>
<value type="str">untested</value>
</answer_choices>
<comment>mount: only root can do that
</comment>
</question>

Then I upgraded to checkbox 0.13.8:

$ dpkg --list 'checkbox*'
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Description
+++-==============-==============-============================================
ii checkbox 0.13.8 System testing application
ii checkbox-cli 0.13.8 Command line interface for checkbox
un checkbox-gtk <none> (no description available)

On rerunning the test command, the result is now passed:

<question name="usb/storage-preinserted">
<answer type="multiple_choice">pass</answer>
<answer_choices>
<value type="str">fail</value>
<value type="str">pass</value>
<value type="str">uninitiated</value>
<value type="str">unresolved</value>
<value type="str">unsupported</value>
<value type="str">untested</value>
</answer_choices>
<comment>--------------------
removable devices currently mounted:
None
removable devices currently not mounted:
/dev/sdb1
--------------------
Now mounting /dev/sdb1 on /tmp/tmpFtzHPt
Found the following mounted ['usb'] partitions:
/dev/sdb1 : /tmp/tmpFtzHPt
--------------------
Running ['usb'] file transfer test for 1 iterations
Creating Temp Data file
File name is :/tmp/tmpBPZSM5
File size is 1048576 bytes
Parent hash is: 6af0821bb5cdb072a93d3fb40b904c59
Copying /tmp/tmpBPZSM5 to /tmp/tmpFtzHPt
Hashing copy on /tmp/tmpFtzHPt
Hash of /tmp/tmpFtzHPt/tmpBPZSM5 on /tmp/tmpFtzHPt is 6af0821bb5cdb072a93d3fb40b904c59
Now umounting /dev/sdb1 on /tmp/tmpFtzHPt
Successfully completed 1 ['usb'] file transfer test iterations
</comment>
</question>

I'll mark this bug verification-done, with the comments that udisks needs to be present (not a problem for ubuntu desktop but may be needed explicitly on server or variants), and the test command needed a minor tweak as it was referring to an incorrect job.

[Steve Langasek (vorlon)]

Hello Daniel, or anyone else affected,

Accepted checkbox into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/checkbox/0.13.9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Daniel Manrique (roadmr)]

Ok redoing the test. Turns out this can be run on a virtual machine (e.g. virtualbox), just make sure to "connect" a USB stick to the virtual machine.

With checkbox 0.13.7:

$ exitESC^H^Hdpkg --list checkbox{,-cli}
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Description
+++-==============-==============-============================================
ii checkbox 0.13.7 System testing application
ii checkbox-cli 0.13.7 Command line interface for checkbox

The storage-preinserted test fails:

Test case results report

a: USB tests
 : Space when finished

Please choose (a/ ): a

USB tests

   usb/insert [pass]
   usb/storage-preinserted [failESC]
 : Space when finished

Please choose ( ):

The submission.xml report shows the cause for the failure, as expected:

<question name="usb/storage-preinserted">
<answer type="multiple_choice">fail</answer>
<comment>mount: only root can do that
</comment>

With checkbox 0.13.9:
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Description
+++-==============-==============-============================================
ii checkbox 0.13.9 System testing application
ii checkbox-cli 0.13.9 Command line interface for checkbox

the test reports as passed from the CLI:

Test case results report

a: USB tests
 : Space when finished

Please choose (a/ ): a

USB tests

   usb/insert [pass]
   usb/storage-preinserted [pass]
 : Space when finished

Please choose ( ):

And the submission.xml shows the pass with details about the testing on the USB stick:

<question name="usb/storage-preinserted">
<answer type="multiple_choice">pass</answer>
<comment>--------------------
removable devices currently mounted:
None
removable devices currently not mounted:
/dev/sdb1
--------------------
Now mounting /dev/sdb1 on /tmp/tmptf7wwt
Found the following mounted ['usb'] partitions:
/dev/sdb1 : /tmp/tmptf7wwt
--------------------
Running ['usb'] file transfer test for 1 iterations
Creating Temp Data file
File name is :/tmp/tmpCFJFdH
File size is 1048576 bytes
Parent hash is: f3c46ef131688ed1aed9a9c91a248028
Copying /tmp/tmpCFJFdH to /tmp/tmptf7wwt
Hashing copy on /tmp/tmptf7wwt
Hash of /tmp/tmptf7wwt/tmpCFJFdH on /tmp/tmptf7wwt is f3c46ef131688ed1aed9a9c91a248028
Now umounting /dev/sdb1 on /tmp/tmptf7wwt
Successfully completed 1 ['usb'] file transfer test iterations
</comment>

I'll set this as verification done.

[Adam Conrad (adconrad)]

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package checkbox - 0.13.9

---------------
checkbox (0.13.9) precise-proposed; urgency=low

  * Updated fix for bug #1026614 which was missing some code.
  * Removed bad fix for bug 990133.

checkbox (0.13.8) precise-proposed; urgency=low

  * New upstream release (LP: #1061198)

  [Marc Tardif]
  * checkbox/dispatcher.py: Updated license to GPLv3 to be consistent with
    rest of Checkbox files (LP: #1049881)

  [Sylvain Pineau]
  * qt/frontend/qtfront.ui: Fixed the test purpose widget size to allow two
    lines of description (LP: #1032255)
  * scripts/camera_test, jobs/camera.txt.in: Added a 10s timeout to the camera
    still test (LP: #990133)

  [Daniel Manrique]
  * jobs/audio.txt.in: replaced gconfaudiosink by autoaudiosink (LP: #978895)
  * Instruct Chromium browser to accept file:// URLs so it can correctly
    open the checkbox submission.xml report (LP: #1026614)
  * scripts/network_check: InvalidURL exception is caught and handled more
    cleanly (LP: #751701)
  * Set the correct user (root) for fwts-wakealarm test (LP: #1004102)
  * Set correct user (root) for usb/storage-preinserted, so it works correctly
    on servers (LP: #1004131)

  [Nathan Williams]
  * scripts/network_check: Fixed exception handling in the absence of zenity
    (LP: #988260)
 -- Daniel Manrique <email address hidden> Tue, 20 Nov 2012 16:31:02 -0500