| 2006-02-03 15:36:04 |
Ian Jackson |
bug |
|
|
added bug |
| 2006-02-03 15:36:04 |
Ian Jackson |
bug |
|
|
assigned to soyuz (upstream) |
| 2006-02-06 11:12:40 |
Daniel Silverstone |
bug |
|
|
added subscriber Launchpad Developers |
| 2006-02-08 12:23:40 |
Diogo Matsubara |
soyuz: severity |
Normal |
Wishlist |
|
| 2006-02-08 12:23:40 |
Diogo Matsubara |
soyuz: statusexplanation |
|
|
|
| 2006-04-22 19:07:11 |
Celso Providelo |
soyuz: status |
Unconfirmed |
Needs Info |
|
| 2006-04-22 19:07:11 |
Celso Providelo |
soyuz: assignee |
|
cprov |
|
| 2006-04-22 19:07:11 |
Celso Providelo |
soyuz: statusexplanation |
|
I agree with some of your points and we do have a plan for rearranging the directory structure on the server to allow 'lazy' uploads and also allow 'Personal Package Archives'uploads to be landed. Something like:
{{{
/distros/<distro_name>/
/people/<person_name>/distros/<distro_name>/
}}}
Each directory beyond those paths will be processed contextually.
Following you suggestion, I think, we need to support a single control file inside the upload directory which controls its processing, of course, it needs support from 'dput' side. It would be something like:
{{{
at /distros/ubuntu/<upload_directory_name>/<control_file> containing:
proccess = False
}}}
I don't know how to define a reliable way to name the upload directory, since it will be required in the' dput' side to continue the upload later.
Also it needs a way to minimize exploits by having a lot of locked upload_directories, maybe removing directories older than a some age (one day would be a lot, IMO).
Anyway, the current prototype hasidentified problem, let's discuss available mid-term solutions. |
|
| 2006-07-07 15:11:20 |
Ian Jackson |
soyuz: status |
Needs Info |
Confirmed |
|
| 2006-07-07 15:11:20 |
Ian Jackson |
soyuz: statusexplanation |
I agree with some of your points and we do have a plan for rearranging the directory structure on the server to allow 'lazy' uploads and also allow 'Personal Package Archives'uploads to be landed. Something like:
{{{
/distros/<distro_name>/
/people/<person_name>/distros/<distro_name>/
}}}
Each directory beyond those paths will be processed contextually.
Following you suggestion, I think, we need to support a single control file inside the upload directory which controls its processing, of course, it needs support from 'dput' side. It would be something like:
{{{
at /distros/ubuntu/<upload_directory_name>/<control_file> containing:
proccess = False
}}}
I don't know how to define a reliable way to name the upload directory, since it will be required in the' dput' side to continue the upload later.
Also it needs a way to minimize exploits by having a lot of locked upload_directories, maybe removing directories older than a some age (one day would be a lot, IMO).
Anyway, the current prototype hasidentified problem, let's discuss available mid-term solutions. |
|
|
| 2010-06-01 16:50:59 |
Curtis Hovey |
soyuz: assignee |
Celso Providelo (cprov) |
|
|
| 2011-02-22 10:22:42 |
Julian Edwards |
tags |
feature lp-soyuz |
feature lp-soyuz poppy |
|
| 2011-05-30 08:30:21 |
William Grant |
removed subscriber Canonical Launchpad Engineering |
|
|
|
| 2015-02-19 00:39:24 |
Colin Watson |
affects |
launchpad |
txpkgupload |
|
| 2015-02-19 00:39:34 |
Colin Watson |
tags |
feature lp-soyuz poppy |
feature lp-soyuz |
|
| 2021-01-15 15:40:00 |
Alvaro Lemus VIveros |
information type |
Public |
Private |
|
| 2021-01-15 15:41:01 |
Alvaro Lemus VIveros |
removed subscriber Christian Reis |
|
|
|
| 2021-01-15 15:41:01 |
Alvaro Lemus VIveros |
removed subscriber EoflaOE |
|
|
|
| 2021-01-15 16:04:54 |
Ian Jackson |
description |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
affects /products/soyuz
done
I am informed that soyuz, when handling incoming uploads, assumes that
all of the components of an upload (the .changes and the files listed
in it) will appear in a single FTP session. This is a violation of
the implied semantics of FTP, and can cause practical problems.
For example, if you use dupload but your upload fails (eg due to
network problems) after successfully completing some files, then
dupload will record success for those files but soyuz will delete
then. If you then rerun dupload it will upload only the files which
were not successfully transferred the first time, but the
already-uploaded files will have been previously deleted by soyuz.
As another example, you might reasonably upload the different parts of
an upload from different systems to save bandwidth on small links.
(Often the .orig.tar.gz is very large.)
As a third example, you might be behind an application relay (web
proxy) which starts a new FTP connection for each transfer. That's
obviously not ideal and is slow and wasteful but it's not demonstrably
wrong.
The correct solution is for soyuz to keep files hanging around rather
than giving each new upload connection a new blank directory. Clashes
between files of different names can be resolved in favour of the most
recent, if each target distribution or namespace has a separate upload
directory. Races can be avoided by careful programming.
Ian.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.6 <http://mailcrypt.sourceforge.net/>
iD8DBQFD43hN8jyP9GfyNQARAqM9AJ4jxO7Jqs/Z+XMX2Khl0YBr3MmvPwCeKSiR
DI+R1DR6Cp52eJfLewbWnNI=
=CRKM
-----END PGP SIGNATURE----- |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
affects /products/soyuz
done
I am informed that soyuz, when handling incoming uploads, assumes that
all of the components of an upload (the .changes and the files listed
in it) will appear in a single FTP session. This is a violation of
the implied semantics of FTP, and can cause practical problems.
For example, if you use dupload but your upload fails (eg due to
network problems) after successfully completing some files, then
dupload will record success for those files but soyuz will delete
then. If you then rerun dupload it will upload only the files which
were not successfully transferred the first time, but the
already-uploaded files will have been previously deleted by soyuz.
As another example, you might reasonably upload the different parts of
an upload from different systems to save bandwidth on small links.
(Often the .orig.tar.gz is very large.)
As a third example, you might be behind an application relay (web
proxy) which starts a new FTP connection for each transfer. That's
obviously not ideal and is slow and wasteful but it's not demonstrably
wrong.
The correct solution is for soyuz to keep files hanging around rather
than giving each new upload connection a new blank directory. Clashes
between files of different names can be resolved in favour of the most
recent, if each target distribution or namespace has a separate upload
directory. Races can be avoided by careful programming.
Ian.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.6 <http://mailcrypt.sourceforge.net/>
iD8DBQFD43hN8jyP9GfyNQARAqM9AJ4jxO7Jqs/Z+XMX2Khl0YBr3MmvPwCeKSiR
DI+R1DR6Cp52eJfLewbWnNI=
=CRKM
-----END PGP SIGNATURE----- |
|
| 2021-01-15 16:07:27 |
Colin Watson |
information type |
Private |
Public |
|
