[FFE] Every script that uses udisk in currently broken in Quantal

Bug #1016035 reported by Sylvain Pineau
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Checkbox
Fix Released
High
Zygmunt Krynicki
checkbox (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

udisks2 will be the default backend for gnome/unity in Quantal:
https://wiki.ubuntu.com/QuantalQuetzal/TechnicalOverview/Alpha1#Ubuntu

The current version works well for udisks version 1 but we should add the support for udisks2 as well:
http://udisks.freedesktop.org/docs/latest/ref-dbus.html
== Importance==

This feature is pretty important to have as currently all the storage tests are broken; they do not work on Quantal and will produce failing results even for users with working hardware. This code enables these features to work correctly under Quantal.

Udisks2 does not provide some bits of information that udisks1 does, and which are necessary to have coverage of all device types that checkbox can test (usb 2.0 and 3.0, memory cards, esata, firewire). So the code is somewhat extensive to account for all these use cases.

A few changes to job definitions were needed (the addition of --memorycard parameter) since it's needed for memory cards to be properly recognized.

== Risk assessment ==

The udisks2 code was developed in an incremental fashion. Due to the
requirement that the same code base keep working on Ubuntu 12.04, the changes
were done carefully to minimize breakage.

Since udisks2 introduces a few API changes and removes certain properties the
old code depended on, particularly speed reporting which is needed for proper
USB 3.0 testing, some refactoring was needed to abstract these changes and keep
the code as clean as possible. Unit tests for these components help ensure
there's no breakage while changing code. While developing, many tests with
actual hardware were performed to ensure that device detection and testing was
reliable and failure scenarios were handled properly.

Once all the parts were ready to land, the udisks2 code was integrated and
merged into checkbox. During this phase the code continued to be tested with
actual hardware to guarantee that all the pieces "fit together" correctly and
the end result worked adequately.

Once everything was in place, two sets of tests were performed: a set of tests
to ensure the features worked as required, and validate that nothing had broken
during final integration. Finally, when preparing the Ubuntu upload, a package
was built and the scripts were tested from the sort of installation an end-user
would see.

Since the code was very extensively tested during development, we believe the
risk of including this to be rather low, and any impact would be localized to
the storage device testing section of checkbox, so the symptoms would in any
case be very easy to identify and clear to solve. Despite this, during testing
all the code behaved properly so if any, the potential remaining problems
should be corner-cases unlikely to have a widespread impact.

== Testing ==

We tested building the package (build log is attached as a comment), installing and upgrading (logs also as comments). We also redid our manual testing by launching the modified scripts (removable_storage_watcher and removable_storage_test) with various parameters (especially the -m and -p parameters used to test for devices with specific speeds, to validate in particular that usb 3.0 devices are testable) and different kinds of devices. The scripts work well and as expected.

Related branches

Revision history for this message
Brendan Donegan (brendan-donegan) wrote : Re: Add udisks2 support to scripts/removable_storage_* scripts

This has reared it's head in weekly testing:

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/dbus/bus.py", line 175, in activate_name_owner return self.get_name_owner(bus_name) File "/usr/lib/python3/dist-packages/dbus/bus.py", line 361, in get_name_owner 's', (bus_name,), **keywords) File "/usr/lib/python3/dist-packages/dbus/connection.py", line 651, in call_blocking message, timeout) dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NameHasNoOwner: Could not get owner of name 'org.freedesktop.UDisks': no such name During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/share/checkbox/scripts/removable_storage_test", line 305, in <module> sys.exit(main()) File "/usr/share/checkbox/scripts/removable_storage_test", line 205, in main test.get_disk_info(args.device) File "/usr/share/checkbox/scripts/removable_storage_test", line 75, in get_disk_info "/org/freedesktop/UDisks") File "/usr/lib/python3/dist-packages/dbus/bus.py", line 241, in get_object follow_name_owner_changes=follow_name_owner_changes) File "/usr/lib/python3/dist-packages/dbus/proxies.py", line 248, in __init__ self._named_service = conn.activate_name_owner(bus_name) File "/usr/lib/python3/dist-packages/dbus/bus.py", line 180, in activate_name_owner self.start_service_by_name(bus_name) File "/usr/lib/python3/dist-packages/dbus/bus.py", line 278, in start_service_by_name 'su', (bus_name, flags))) File "/usr/lib/python3/dist-packages/dbus/connection.py", line 651, in call_blocking message, timeout) dbus.exceptions.DBusException: org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.UDisks was not provided by any .service files

Changed in checkbox:
status: New → Confirmed
importance: Undecided → High
summary: - Add udisks2 support to scripts/removable_storage_watcher
+ Add udisks2 support to scripts/removable_storage_* scripts
Zygmunt Krynicki (zyga)
Changed in checkbox:
assignee: nobody → Zygmunt Krynicki (zkrynicki)
Changed in checkbox:
status: Confirmed → In Progress
Ara Pulido (ara)
Changed in checkbox:
milestone: none → 0.14.x
Revision history for this message
Zygmunt Krynicki (zyga) wrote :

So, some analysis:

On quantal, udisks (the udisks 1) package is installed by default, most likely it is pulled by checkbox itself. We should check with the platform team if they are happy to ship both by default on the CD.

The specific test, usb/insert, that runs 'removable_storage_watcher insert usb' fails. Quick inspection suggests that the second stage of detection (the act of mounting the filesystem) is not reported by udisks. Perhaps it has been moved to udisks2 as there are lots of different events generated on that interface.

To be precise the following events are reported when a device is inserted to a quantal system (see attached log file for details):

1) path=/org/freedesktop/UDisks; interface=org.freedesktop.UDisks; member=DeviceAdded object path "/org/freedesktop/UDisks/devices/sdb"
2) the equivalent, very long signal for udisk2
3) The udisks1 signal for /dev/sdb1 (the first partition)
4) The equivalent, very long signal for udisks2

Note that the filesystem is mounted at the end of this operation but there are no signals being emitted that indicate this.

Revision history for this message
Zygmunt Krynicki (zyga) wrote :

After discussing this with pitti on #ubuntu-devel I know what is happening and how to address that:

1) On quantal udisks1 not running by default and is only service-activated by checkbox
2) On quantal udisks1 is not handling filesystem mounts as those are requested by gvfs and gvfs is now using udisks2 API. In consequence we must address this if we still wish to ship the code that requires udisks1 to keep minimum support
3) There is a strong desire to drop udisks1 from the ISO. Apart from checkbox there is only one other package that requires it in the default install, usb-creator-common. It is possible that it will go away but it is not certain.

I would suggest the following actions to take place:

1) We should add optional udisks2 support to all the places that used udisk1 in the past (this should be a separate bug)
2) We should alter our packaging so that quantal does not require udisks anymore
3) We should auto-detect if udisks2 is available and prefer it automatically
4) We should reconsider if our tests should check for device insertion or actual filesystem mounting. Since we require a mounted filesystem in other tests then I'd suggest at least improving the error message.

Revision history for this message
Zygmunt Krynicki (zyga) wrote :

Just for the record we're moving to full UDisks2 support

Ara Pulido (ara)
Changed in checkbox:
importance: High → Critical
Ara Pulido (ara)
Changed in checkbox:
importance: Critical → High
Zygmunt Krynicki (zyga)
Changed in checkbox:
status: In Progress → Fix Committed
Ara Pulido (ara)
summary: - Add udisks2 support to scripts/removable_storage_* scripts
+ Every script that uses udisk in currently broken in Quantal
Revision history for this message
Daniel Manrique (roadmr) wrote : Re: Every script that uses udisk in currently broken in Quantal

Attached is a build log for this checkbox package, produced with:

sbuild -A --arch=amd64 -d quantal checkbox_0.14.9.dsc

Revision history for this message
Daniel Manrique (roadmr) wrote :

This is the upgrade log for a system with checkbox 0.14.7, I upgraded only checkbox and checkbox-qt which are in the default Ubuntu installation:

(Reading database ... 159405 files and directories currently installed.)
Preparing to replace checkbox 0.14.7 (using checkbox_0.14.9_amd64.deb) ...
Unpacking replacement checkbox ...
Preparing to replace checkbox-qt 0.14.7 (using checkbox-qt_0.14.9_amd64.deb) ...
Unpacking replacement checkbox-qt ...
Setting up checkbox (0.14.9) ...
Processing triggers for man-db ...
Processing triggers for hicolor-icon-theme ...
Setting up checkbox-qt (0.14.9) ...
Processing triggers for desktop-file-utils ...
Processing triggers for bamfdaemon ...
Rebuilding /usr/share/applications/bamf.index...
Processing triggers for gnome-menus ...

Revision history for this message
Daniel Manrique (roadmr) wrote :

Install log on a system with no checkbox packages installed (I purged them prior to trying this installation):

Selecting previously unselected package checkbox.
(Reading database ... 159012 files and directories currently installed.)
Unpacking checkbox (from checkbox_0.14.9_amd64.deb) ...
Selecting previously unselected package checkbox-qt.
Unpacking checkbox-qt (from checkbox-qt_0.14.9_amd64.deb) ...
Setting up checkbox (0.14.9) ...
Processing triggers for man-db ...
Processing triggers for hicolor-icon-theme ...
Setting up checkbox-qt (0.14.9) ...
Processing triggers for desktop-file-utils ...
Processing triggers for bamfdaemon ...
Rebuilding /usr/share/applications/bamf.index...
Processing triggers for gnome-menus ...

Daniel Manrique (roadmr)
description: updated
summary: - Every script that uses udisk in currently broken in Quantal
+ [FFe] Every script that uses udisk in currently broken in Quantal
summary: - [FFe] Every script that uses udisk in currently broken in Quantal
+ [FFE] Every script that uses udisk in currently broken in Quantal
Daniel Manrique (roadmr)
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package checkbox - 0.14.9

---------------
checkbox (0.14.9) quantal; urgency=low

  * New upstream release (LP: #1064504)

  [Zygmunt Krynicki]
  * checkbox/dbus/__init__.py, checkbox/dbus/udisks2.py, checkbox/heuristics/,
    checkbox/heuristics/__init__.py, checkbox/heuristics/udev.py,
    checkbox/heuristics/udisks2.py, checkbox/tests/heuristics.py,
    checkbox/udev.py, checkbox/parsers/udevadm.py, debian/control,
    jobs/mediacard.txt.in, scripts/removable_storage_test,
    scripts/removable_storage_watcher, setup.py: Added udisks2 support
    (LP: #1016035)
 -- Daniel Manrique <email address hidden> Tue, 09 Oct 2012 12:23:41 -0400

Changed in checkbox (Ubuntu):
status: New → Fix Released
Changed in checkbox:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.