Add helper function to fetch many files at once

Bug #450629 reported by Free Ekanayaka on 2009-10-13
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Landscape Client
Medium
Free Ekanayaka
landscape-client (Ubuntu)
Undecided
Unassigned
Intrepid
Undecided
Unassigned
Jaunty
Undecided
Unassigned
Karmic
Undecided
Unassigned
Lucid
Undecided
Unassigned

Bug Description

We currently have a fetch_async function for downloading a single URL. It would be nice to have one to retrieve a list of URLs, and have it callback once all of them have completed. In particular this feature will be used to fetch the files of the release-ugprade tool used for dist-upgrades.

The attached branch implements the proposed behaviour, and also add a helper function to parse the /etc/lsb-release, which is used in a couple of spots already, and will be used in the release-upgrade feature as well.

Changed in landscape-client:
status: New → In Progress
importance: Undecided → Medium
assignee: nobody → Free Ekanayaka (free.ekanayaka)
milestone: none → 1.4.0
tags: added: review
Sidnei da Silva (sidnei) wrote :

[1] parse_lsb_release: Please put the fd.close() in a 'finally:' statement.

[2] fetch_to_files: If the url ends with a '/', the filename will be an empty string:

>>> url = "X/"
>>> url.split("/")[-1]
''

[3] test_fetch_many_async: s/retrives/retrieves

Looks good! +1.

Thomas Herve (therve) wrote :

[1]
+ def completed(result):
+ self.fail()
+
+ def aborted(failure):
+ self.assertEquals(fetched_urls, ["http://right/", "http://wrong/"])
+
+ d.addCallback(completed)
+ d.addErrback(aborted)
+ return d

You should use assertFailure in this case: you can remove the self.fail(), and add aborted as a callback.

[2]
+ L{fetch_to_files} fetches a list of URLs and save they're content

Typo: their content

[3] Pyflakes:
landscape/package/tests/test_taskhandler.py:7: 'CommandError' imported but unused

+1!

tags: removed: review
Free Ekanayaka (free.ekanayaka) wrote :

Thanks Thomas and Sidnei! All comments addressed.

Jamu Kakar (jkakar) on 2009-10-20
Changed in landscape-client:
milestone: 1.4.0 → 1.4.1
Changed in landscape-client:
status: In Progress → Fix Committed
tags: added: needs-testing
tags: removed: needs-testing
Jamu Kakar (jkakar) on 2009-12-16
tags: added: 1.5-upgrade-client
Changed in landscape-client (Ubuntu Lucid):
status: New → Fix Committed
Changed in landscape-client (Ubuntu Lucid):
status: Fix Committed → Fix Released

Accepted landscape-client into intrepid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in landscape-client (Ubuntu Intrepid):
status: New → Fix Committed
tags: added: verification-needed
Changed in landscape-client (Ubuntu Jaunty):
status: New → Fix Committed
Martin Pitt (pitti) wrote :

Accepted landscape-client into jaunty-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in landscape-client (Ubuntu Karmic):
status: New → Fix Committed
Martin Pitt (pitti) wrote :

Accepted landscape-client into karmic-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package landscape-client - 1.4.0-0ubuntu0.8.10.0

---------------
landscape-client (1.4.0-0ubuntu0.8.10.0) intrepid-proposed; urgency=low

  * New upstream release (LP: #497351)

  * Bug fixes:
    - Fix landscape daemons fail to start when too many groups are
      available (LP: #456124)
    - Fix landscape programs wake up far too much. (LP: #340843)
    - Fix Package manager fails with 'no such table: task' (LP #465846)
    - Fix test suite leaving temporary files around (LP #476418)
    - Fix the 1hr long wait for user data to be uploaded following a
      resynchronisation (LP #369000)

  * Add support for Ubuntu release upgrades:
    - Add helper function to fetch many files at once (LP: #450629)
    - Handle release-upgrade messages in the packagemanager
      plugin (LP: #455217)
    - Add a release-upgrader task handler (LP: #462543)
    - Support upgrade-tool environment variables (LP: #463321)

  * Add initial support for Smart package locking:
    - Detect and report changes about Smart package locks (#488108)

  * Packaging fixes:
    - Turn unnecessary Pre-Depends on python-gobject into a regular Depends
    - If it's empty, remove /etc/landscape upon purge
 -- Free Ekanayaka <email address hidden> Wed, 16 Dec 2009 10:50:05 +0100

Changed in landscape-client (Ubuntu Intrepid):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package landscape-client - 1.4.0-0ubuntu0.9.04.0

---------------
landscape-client (1.4.0-0ubuntu0.9.04.0) jaunty-proposed; urgency=low

  * New upstream release (LP: #497351)

  * Bug fixes:
    - Fix landscape daemons fail to start when too many groups are
      available (LP: #456124)
    - Fix landscape programs wake up far too much. (LP: #340843)
    - Fix Package manager fails with 'no such table: task' (LP #465846)
    - Fix test suite leaving temporary files around (LP #476418)
    - Fix the 1hr long wait for user data to be uploaded following a
      resynchronisation (LP #369000)

  * Add support for Ubuntu release upgrades:
    - Add helper function to fetch many files at once (LP: #450629)
    - Handle release-upgrade messages in the packagemanager
      plugin (LP: #455217)
    - Add a release-upgrader task handler (LP: #462543)
    - Support upgrade-tool environment variables (LP: #463321)

  * Add initial support for Smart package locking:
    - Detect and report changes about Smart package locks (#488108)

  * Packaging fixes:
    - Turn unnecessary Pre-Depends on python-gobject into a regular Depends
    - If it's empty, remove /etc/landscape upon purge
 -- Free Ekanayaka <email address hidden> Wed, 16 Dec 2009 10:50:05 +0100

Changed in landscape-client (Ubuntu Jaunty):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package landscape-client - 1.4.0-0ubuntu0.9.10.0

---------------
landscape-client (1.4.0-0ubuntu0.9.10.0) karmic-proposed; urgency=low

  * New upstream release (LP: #497351)

  * Bug fixes:
    - Fix landscape daemons fail to start when too many groups are
      available (LP: #456124)
    - Fix landscape programs wake up far too much. (LP: #340843)
    - Fix Package manager fails with 'no such table: task' (LP #465846)
    - Fix test suite leaving temporary files around (LP #476418)
    - Fix the 1hr long wait for user data to be uploaded following a
      resynchronisation (LP #369000)

  * Add support for Ubuntu release upgrades:
    - Add helper function to fetch many files at once (LP: #450629)
    - Handle release-upgrade messages in the packagemanager
      plugin (LP: #455217)
    - Add a release-upgrader task handler (LP: #462543)
    - Support upgrade-tool environment variables (LP: #463321)

  * Add initial support for Smart package locking:
    - Detect and report changes about Smart package locks (#488108)

  * Packaging fixes:
    - Turn unnecessary Pre-Depends on python-gobject into a regular Depends
    - If it's empty, remove /etc/landscape upon purge
 -- Free Ekanayaka <email address hidden> Wed, 16 Dec 2009 10:50:05 +0100

Changed in landscape-client (Ubuntu Karmic):
status: Fix Committed → Fix Released
Changed in landscape-client:
status: Fix Committed → Fix Released
tags: removed: verification-needed
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers