Verify Networks fails with unusable error message
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Fuel for OpenStack |
Fix Released
|
High
|
Vladimir Sharshov | ||
6.1.x |
Won't Fix
|
Medium
|
Denis Puchkin |
Bug Description
Steps, this a new Fuel Master using Fuel 6.1
1. HA Environment
2. 3 Controllers, 6 Compute Nodes, and 6 Ceph Nodes
3. Run Verify Networks before Deploying (have not deployed yet)
Verify Networks Runs (for quite a long time, it should have a 5 minute timeout)
Then it fails with an error message and says to look in the Astute Log:
This is what the Astute Log says:
2015-07-03 03:49:18 ERR
[594] Error running RPC method verify_networks: c7c32b04-
ID: 39 - Reason: execution expired
ID: 33 - Reason: execution expired
ID: 5 - Reason: execution expired
ID: 1 - Reason: execution expired
ID: 55 - Reason: execution expired
ID: 54 - Reason: execution expired
ID: 44 - Reason: execution expired
ID: 36 - Reason: execution expired
ID: 8 - Reason: execution expired
ID: 2 - Reason: execution expired
ID: 4 - Reason: execution expired
ID: 7 - Reason: execution expired
ID: 3 - Reason: execution expired
ID: 35 - Reason: execution expired
ID: 53 - Reason: execution expired
, trace:
["/usr/
"/usr/
"/usr/
"/usr/
"/usr/
"/usr/
"/usr/
"/usr/
"/usr/
"/usr/
"/usr/
"/usr/
"/usr/
"/usr/
"/usr/
"/usr/
"/usr/
Reason:
This error is because there is no connectivity to the repositories during network checker which means there is probably a misconfiguration in the networking. What is happening is that network checker is configuring the network interfaces as part and then attempts to request all the repositories defined for the environment. Unfortunately there is not timeout on the urlaccesscheck when it goes to make the url request.
The default we use on the fuel-library side for similar functionaility after we run l23network is to wait at most 180 seconds per url and at most a 60 second open timeout and a 60 second read timeout.
Because there is no timeout on the urlaccesschecker it's using the defaults which is greater than the timeout on the mcollective task
Workaround:
Go to the Settings tab and remove repos that aren't available from the nodes.
Changed in fuel: | |
assignee: | Fuel Library Team (fuel-library) → Fuel Python Team (fuel-python) |
status: | New → Confirmed |
importance: | Medium → High |
tags: | added: customer-found |
tags: | added: module-astute |
Changed in fuel: | |
assignee: | Fuel Python Team (fuel-python) → Vladimir Sharshov (vsharshov) |
tags: | added: tricky |
Changed in fuel: | |
status: | In Progress → Fix Committed |
tags: | added: on-verification |
tags: | added: support |
description: | updated |
tags: | added: wontfix-low |
The Fuel 6.0 error log gave a much better error message.