the script will be split  to the following components:

file-transfer protocol
=================

incapsulates the protocols, that are supported to transfer file.

.. code:: python

  @six.add_metaclass(abc.ABCMeta)
  class Transport(object):
      @abc.abstractmethod
      def glob(self, path, mask=None):
          """enumerate files in patch
          returns list of pairs (isdir, name)
          """

      @abc.abstractmethod
      def exists(self, path):
          """checks that path exists"""

      @abc.abstractmethod
      def fetch(self, src, dst):
          """copies file from src to dst"""


protocols are planned to support:  rsync,  http[1]


repositories
==========

contains functional to work with repository

.. code:: python
   class Package:
          name # package name
          version = # package version
          requires = []  #list of required packages
          repository  # the package repository

  @six.add_metaclass(abc.ABCMeta)
  class Repository(object):
      @abc.abstractmethod
      def get_packages(self, filter=None):
          """get list of all packages."""

      @abc.abstractmethod
      def clone(self, path):
          """clones repository to local fs"""

      @abc.abstractmethod
      def replicate(self, package, mirror):
          """replicates package to mirror"""
     
  @six.add_metaclass(abc.ABCMeta)
class Mirror(object):
      @abc.abstractmethod
       def get_path(package):
            """get path for package"""
    
      @abc.abstractmethod
       def is_dirty(package):
            """checks that re-sync is required"""
      

repositories are planned to support:  ubuntu - repository, yum repositories[1]


modules:

command
========

provides function to launch sub-commands and dry-run semantic

.. code:: python

def call(cmd, **kwargs):
   """for details see subprocess.call."""

def check_call(cmd, **kwargs):
   """for details see subprocess.call."""

def check_output(cmd, **kwargs):
   """for details see subprocess.call."""

def find_executables(executable):
   """get the fullpath for executable"""


createmirror:

.. code:: python

  def resolve(requires, repositories, dependencies):
        """resolves list of requires recursively."""



workflow:
  repositories = []
  for r in config.repositories:
     repositories.append(open_repository(r))

 packages = defaultdict()  # repository -> set of packages

 for r in config.target_repository:
        for package in r.get_packages():
              # it is possible to start synchronisation mirror in parallel
              packages[r].add(package)
              resolve_requires(package.requires, repositories, packages)

local_repositories = []

for repository, packages in packages:
    mirror = repository.clone(config.target)
   # this part of code can be run in parallel
    for p in packages:
        repository.replicate(package, mirror)
   mirror.update()e
   local_repositories.append(mirror.get_path)

# dump information about local repositories.
# update fuel-main if required

configuration file:

.. code:: yaml

commands:
  dry_run: False
  retries: 1
  retry_delay: 1

logging:
  filename: /var/log/fuel-mirror.log
  level: debug

transport:
   proxy: ""
  
repositories:
  -  os:
        ubuntu:
            main:
              - http://mirror.yandex.ru/ubuntu/dists/trusty
            updates:
              - http://mirror.yandex.ru/ubuntu/dists/trusty-updates
            security:
              - http://mirror.yandex.ru/ubuntu/dists/trusty-security
        centos:
             main:
              - http://mirror.yandex.ru/ubuntu/dists/trusty   
  - mos:
        ubuntu:
            main:
              - http://mirror.yandex.ru/ubuntu/dists/trusty
            updates:
              - http://mirror.yandex.ru/ubuntu/dists/trusty-updates
            security:
              - http://mirror.yandex.ru/ubuntu/dists/trusty-security
        centos:
           - http://mirror.yandex.ru/centos/dists/


command line options:
--dry-run
--repository=<path to repository>  (os|os.ubuntu|os.ubuntu.main)
--proxy=10.250.1.1
...


[1] will be implemented as separate task.