Verification failed. Method verify_networks. 768cc4a8-1adb-43d4-8432-ecad9efd9ea7: MCollective call failed in agent 'net_probe', method 'start_frame_listeners', failed nodes:

[Anudeep]

Hello all,

Detailed bug description:
Description of the environment:

I am trying to deploy MOS 9.0 on bare metal

3- controllers
15- computes
1- cinder (pure storage)

Detailed bug description:
Network verification failed.
Fuel master node can able to ping slave node.
The service mcollective is in active state.
I tried to reset slave nodes but still throws the same bug.

Astute logs
 [1602] Error running RPC method verify_networks: 768cc4a8-1adb-43d4-8432-ecad9efd9ea7: MCollective call failed in agent 'net_probe', method 'start_frame_listeners', failed nodes:
ID: 2 - Reason: Listener did not reported status
ID: 1 - Reason: Listener did not reported status
, trace:
["/usr/share/gems/gems/astute-9.0.0/lib/astute/mclient.rb:128:in `check_results_with_retries'",
 "/usr/share/gems/gems/astute-9.0.0/lib/astute/mclient.rb:70:in `method_missing'",
 "/usr/share/gems/gems/astute-9.0.0/lib/astute/network.rb:163:in `start_frame_listeners'",
 "/usr/share/gems/gems/astute-9.0.0/lib/astute/network.rb:52:in `check_network'",
 "/usr/share/gems/gems/astute-9.0.0/lib/astute/orchestrator.rb:180:in `verify_networks'",
 "/usr/share/gems/gems/astute-9.0.0/lib/astute/server/dispatcher.rb:143:in `verify_networks'",
 "/usr/share/gems/gems/astute-9.0.0/lib/astute/server/server.rb:172:in `dispatch_message'",
 "/usr/share/gems/gems/astute-9.0.0/lib/astute/server/server.rb:131:in `block in dispatch'",
 "/usr/share/gems/gems/astute-9.0.0/lib/astute/server/task_queue.rb:64:in `call'",
 "/usr/share/gems/gems/astute-9.0.0/lib/astute/server/task_queue.rb:64:in `block in each'",
 "/usr/share/gems/gems/astute-9.0.0/lib/astute/server/task_queue.rb:56:in `each'",
 "/usr/share/gems/gems/astute-9.0.0/lib/astute/server/task_queue.rb:56:in `each'",
 "/usr/share/gems/gems/astute-9.0.0/lib/astute/server/server.rb:128:in `each_with_index'",
 "/usr/share/gems/gems/astute-9.0.0/lib/astute/server/server.rb:128:in `dispatch'",
 "/usr/share/gems/gems/astute-9.0.0/lib/astute/server/server.rb:106:in `block in perform_main_job'"]

Expected results:
Deploy successfully

Actual results:
Can't able to deploy

Please, help me is getting the environment deployed successfully.
Thanks in advance
Anudeep

[Stanislaw Bogatkin (sbogatkin)]

This bug just can't be solved without diagnostic snapshot, cause there is no info about network configuration for this env. Please, attach diag snapshot.

[Anudeep (anudeep1248)]

Thanks for looking in to this issue.
I tried to upload my diagnostic snapshot but, the web page for showing time-out. I think as it takes more time to upload file.

Here is the link where you can access the diagnostic report

https://www.dropbox.com/s/w3mshz7zrivq85f/fuel-snapshot-2017-01-11_14-18-32.tar?dl=0

Thanks,
Anudeep

[Stanislaw Bogatkin (sbogatkin)]

Looking at logs it seems that both of yours slave nodes has 2 interfaces - eno1 and eno2. You assigned each interface on each node to some network, but your eno2 interface on each node seems not to be physically connected to any hardware. As a result, network checker cannot reach these interfaces to the UP state and fails.

Check if your interfaces really connected to some hardware or get them unassigned from the networks your cluster gonna use.

[Oleksiy Molchanov (omolchanov)]

Marking as Invalid, because of no activity for more than a month.