ERR after adding controllers. (/Stage[main]/Apt::Update/Exec[apt_update]) Failed to call refresh: /usr/bin/apt-get update returned 100 instead of one of [0]

[Anastasia Palkina]

1. Create new environment by default
2. Add 1 controller
3. Start deployment. It was successful
4. Add 2 controllers
5. Start deployment. It was successful
6. But there is error on third controller (node-10):

2016-01-13 13:10:50 ERR (/Stage[main]/Apt::Update/Exec[apt_update]) Failed to call refresh: /usr/bin/apt-get update returned 100 instead of one of [0]

Logs are here: https://drive.google.com/a/mirantis.com/file/d/0B6SjzarTGFxaaVZaU1FHOVZUSjg/view?usp=sharing

VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  api: "1.0"
  build_number: "427"
  build_id: "427"
  fuel-nailgun_sha: "9ebbaa0473effafa5adee40270da96acf9c7d58a"
  python-fuelclient_sha: "4f234669cfe88a9406f4e438b1e1f74f1ef484a5"
  fuel-agent_sha: "df16d41cd7a9445cf82ad9fd8f0d53824711fcd8"
  fuel-nailgun-agent_sha: "92ebd5ade6fab60897761bfa084aefc320bff246"
  astute_sha: "c7ca63a49216744e0bfdfff5cb527556aad2e2a5"
  fuel-library_sha: "fae42170a54b98d8e8c8db99b0fbb312633c693c"
  fuel-ostf_sha: "214e794835acc7aa0c1c5de936e93696a90bb57a"
  fuel-mirror_sha: "b62f3cce5321fd570c6589bc2684eab994c3f3f2"
  fuelmenu_sha: "85de57080a18fda18e5325f06eaf654b1b931592"
  shotgun_sha: "63645dea384a37dde5c01d4f8905566978e5d906"
  network-checker_sha: "9f0ba4577915ce1e77f5dc9c639a5ef66ca45896"
  fuel-upgrade_sha: "616a7490ec7199f69759e97e42f9b97dfc87e85b"
  fuelmain_sha: "e8e36cff332644576d7853c80b8a53d5b955420a"

[Dmitry Bilunov (dbilunov)]

[apt_update]/returns (notice): W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty-security/main/binary-amd64/Packages Hash Sum mismatch
[apt_update]/returns (notice):
[apt_update]/returns (notice): W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty-security/universe/binary-amd64/Packages Hash Sum mismatch
[apt_update]/returns (notice): E: Some index files failed to download. They have been ignored, or old ones used instead.
[apt_update] (err): Failed to call refresh: /usr/bin/apt-get update returned 100 instead of one of [0]
[apt_update] (err): /usr/bin/apt-get update returned 100 instead of one of [0]

It's either a network issue, or Package(.gz/.bz2) files have been updated on the mirror (a race condition issue).

[slava valyavskiy (slava-val-al)]

@Anastasia, Did you run OSTF tests on affected environment ? Was it successful?

[Anastasia Palkina (apalkina)]

@Slava,
yes, I run OSTF tests and they were successful.

Also I added 2 new controllers and one of them also had this error.
Logs lost because my computer didn't sustain the load and master node dead.