the script will be split to the following components:
file-transfer protocol
=================
incapsulates the protocols, that are supported to transfer file.
.. code:: python
@six.add_metaclass(abc.ABCMeta)
class Transport(object):
@abc.abstractmethod
def glob(self, path, mask=None): """enumerate files in patch
returns list of pairs (isdir, name)
"""
@abc.abstractmethod
def exists(self, path):
"""checks that path exists"""
@abc.abstractmethod
def fetch(self, src, dst):
"""copies file from src to dst"""
protocols are planned to support: rsync, http[1]
repositories
==========
contains functional to work with repository
.. code:: python
class Package:
name # package name
version = # package version
requires = [] #list of required packages repository # the package repository
@six.add_metaclass(abc.ABCMeta)
class Repository(object):
@abc.abstractmethod
def get_packages(self, filter=None):
"""get list of all packages."""
@abc.abstractmethod
def clone(self, path):
"""clones repository to local fs"""
@abc.abstractmethod
def replicate(self, package, mirror): """replicates package to mirror"""
@six.add_metaclass(abc.ABCMeta)
class Mirror(object):
@abc.abstractmethod
def get_path(package):
"""get path for package"""
@abc.abstractmethod
def is_dirty(package): """checks that re-sync is required"""
repositories are planned to support: ubuntu - repository, yum repositories[1]
modules:
command
========
provides function to launch sub-commands and dry-run semantic
.. code:: python
def call(cmd, **kwargs):
"""for details see subprocess.call."""
def check_call(cmd, **kwargs):
"""for details see subprocess.call."""
def check_output(cmd, **kwargs):
"""for details see subprocess.call."""
def find_executables(executable):
"""get the fullpath for executable"""
createmirror:
.. code:: python
def resolve(requires, repositories, dependencies):
"""resolves list of requires recursively."""
workflow:
repositories = []
for r in config.repositories:
repositories.append(open_repository(r))
packages = defaultdict() # repository -> set of packages
for r in config.target_repository:
for package in r.get_packages():
# it is possible to start synchronisation mirror in parallel packages[r].add(package) resolve_requires(package.requires, repositories, packages)
local_repositories = []
for repository, packages in packages:
mirror = repository.clone(config.target)
# this part of code can be run in parallel
for p in packages: repository.replicate(package, mirror)
mirror.update()e
local_repositories.append(mirror.get_path)
# dump information about local repositories.
# update fuel-main if required
the script will be split to the following components:
file-transfer protocol
=================
incapsulates the protocols, that are supported to transfer file.
.. code:: python
@six. add_metaclass( abc.ABCMeta) abc.abstractmet hod
"""enumerate files in patch
class Transport(object):
@
def glob(self, path, mask=None):
returns list of pairs (isdir, name)
"""
@ abc.abstractmet hod
def exists(self, path):
"""checks that path exists"""
@ abc.abstractmet hod
def fetch(self, src, dst):
"""copies file from src to dst"""
protocols are planned to support: rsync, http[1]
repositories
==========
contains functional to work with repository
.. code:: python
repository # the package repository
class Package:
name # package name
version = # package version
requires = [] #list of required packages
@six. add_metaclass( abc.ABCMeta) abc.abstractmet hod
class Repository(object):
@
def get_packages(self, filter=None):
"""get list of all packages."""
@ abc.abstractmet hod
def clone(self, path):
"""clones repository to local fs"""
@ abc.abstractmet hod
"""replicate s package to mirror"""
def replicate(self, package, mirror):
@six. add_metaclass( abc.ABCMeta) abc.abstractmet hod
class Mirror(object):
@
def get_path(package):
"""get path for package"""
@ abc.abstractmet hod
"" "checks that re-sync is required"""
def is_dirty(package):
repositories are planned to support: ubuntu - repository, yum repositories[1]
modules:
command
========
provides function to launch sub-commands and dry-run semantic
.. code:: python
def call(cmd, **kwargs):
"""for details see subprocess.call."""
def check_call(cmd, **kwargs):
"""for details see subprocess.call."""
def check_output(cmd, **kwargs):
"""for details see subprocess.call."""
def find_executable s(executable) :
"""get the fullpath for executable"""
createmirror:
.. code:: python
def resolve(requires, repositories, dependencies):
"""resolves list of requires recursively."""
workflow: repositories: es.append( open_repository (r))
repositories = []
for r in config.
repositori
packages = defaultdict() # repository -> set of packages
for r in config. target_ repository:
packages[ r].add( package)
resolve_ requires( package. requires, repositories, packages)
for package in r.get_packages():
# it is possible to start synchronisation mirror in parallel
local_repositories = []
for repository, packages in packages: clone(config. target)
repository. replicate( package, mirror) repositories. append( mirror. get_path)
mirror = repository.
# this part of code can be run in parallel
for p in packages:
mirror.update()e
local_
# dump information about local repositories.
# update fuel-main if required
configuration file:
.. code:: yaml
commands:
dry_run: False
retries: 1
retry_delay: 1
logging: fuel-mirror. log
filename: /var/log/
level: debug
transport:
proxy: ""
repositories: mirror. yandex. ru/ubuntu/ dists/trusty
updates: mirror. yandex. ru/ubuntu/ dists/trusty- updates
security: mirror. yandex. ru/ubuntu/ dists/trusty- security mirror. yandex. ru/ubuntu/ dists/trusty mirror. yandex. ru/ubuntu/ dists/trusty
updates: mirror. yandex. ru/ubuntu/ dists/trusty- updates
security: mirror. yandex. ru/ubuntu/ dists/trusty- security mirror. yandex. ru/centos/ dists/
- os:
ubuntu:
main:
- http://
- http://
- http://
centos:
main:
- http://
- mos:
ubuntu:
main:
- http://
- http://
- http://
centos:
- http://
command line options: ubuntu| os.ubuntu. main)
--dry-run
--repository=<path to repository> (os|os.
--proxy=10.250.1.1
...
[1] will be implemented as separate task.