Openstack logs from node are not available on UI

Bug #1401852 reported by Tatyanka
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Released
High
Evgeniy L
6.0.x
Fix Committed
High
Evgeniy L
6.1.x
Fix Released
High
Evgeniy L

Bug Description

{"build_id": "2014-12-09_22-41-06", "ostf_sha": "a9afb68710d809570460c29d6c3293219d3624d4", "build_number": "49", "auth_required": true, "api": "1.0", "nailgun_sha": "22bd43b89a17843f9199f92d61fc86cb0f8772f1", "production": "docker", "fuelmain_sha": "3aab16667f47dd8384904e27f70f7a87ba15f4ee", "astute_sha": "16b252d93be6aaa73030b8100cf8c5ca6a970a91", "feature_groups": ["mirantis"], "release": "6.0", "release_versions": {"2014.2-6.0": {"VERSION": {"build_id": "2014-12-09_22-41-06", "ostf_sha": "a9afb68710d809570460c29d6c3293219d3624d4", "build_number": "49", "api": "1.0", "nailgun_sha": "22bd43b89a17843f9199f92d61fc86cb0f8772f1", "production": "docker", "fuelmain_sha": "3aab16667f47dd8384904e27f70f7a87ba15f4ee", "astute_sha": "16b252d93be6aaa73030b8100cf8c5ca6a970a91", "feature_groups": ["mirantis"], "release": "6.0", "fuellib_sha": "2c99931072d951301d395ebd5bf45c8d401301bb"}}}, "fuellib_sha": "2c99931072d951301d395ebd5bf45c8d401301bb"}

Steps to Reroduce:
1. Deploy Ha ubuntu with neutron (it is not critical, because reproduce on each cluster )
2. 3 controller + 2 compute
3. Deploy cluster
4. When cluster ready , run ostf
5. Navigate to the Logs tab, select Logs: Orher Node:and some node , Source: Any openstack source, and any log level

Actual Result:
Openstack logs are unavailable
We send next request:
Remote Address:10.120.0.2:8000
Request URL:http://10.120.0.2:8000/api/logs?node=4&source=os%2Fnova-conductor&level=INFO&from=0&to=78535&_=1418383854629
Request Method:GET
Status Code:200 OK
Request Headersview source
Accept:application/json, text/javascript, */*; q=0.01
Accept-Encoding:gzip,deflate,sdch
Accept-Language:ru-RU,ru;q=0.8,en-US;q=0.6,en;q=0.4
Connection:keep-alive
Cookie:i18next=en-US; token=75681422e0294565aeaf856e4bcb0098
Host:10.120.0.2:8000
Referer:http://10.120.0.2:8000/
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.111 Safari/537.36
X-Auth-Token:75681422e0294565aeaf856e4bcb0098
X-Requested-With:XMLHttpRequest
Query String Parametersview sourceview URL encoded
node:4
source:os/nova-conductor
level:INFO
from:0
to:78535
_:1418383854629
Response Headersview source
Cache-Control:store, no-cache, must-revalidate, post-check=0, pre-check=0
Connection:keep-alive
Content-Encoding:gzip
Content-Type:application/json
Date:Fri, 12 Dec 2014 11:33:13 GMT
Expires:Thu, 01 Jan 1970 00:00:00 GMT
Pragma:no-cache
Server:nginx
Transfer-Encoding:chunked

and response from api is empty:

{"has_more": true, "to": 78708, "from": 0, "entries": []}

Revision history for this message
Tatyanka (tatyana-leontovich) wrote :
Revision history for this message
Tatyanka (tatyana-leontovich) wrote :
description: updated
Revision history for this message
Dmitry Pyzhov (dpyzhov) wrote :

Logs are sent in another log format. Some logs are good: heat-manage.log, heat-api-cfn.log, heat-api.log, heat-api-cloudwatch.log, glance-manage.log, glance-registry.log, glance-api.log, neutron-server.log, neutron-openvswitch-agent.log, neutron-l3-agent.log, neutron-netns-cleanup.log, neutron-metadata-agent.log. For example:
==> ./neutron-metadata-agent.log <==
2014-12-12T11:00:30.098105+00:00 info: 2014-12-12 11:00:30.003 14349 INFO neutron.common.config [-] Logging enabled!

Other logs are sent in another format cannot be parsed by Nailgun. For example:
==> ./nova-api.log <==
2014-12-12T10:57:15.437488+00:00 debug: Loading app ec2 from /etc/nova/api-paste.ini

Revision history for this message
Dmitry Pyzhov (dpyzhov) wrote :

It needs to be fixed either on library or on mos side.

Changed in fuel:
assignee: Fuel Python Team (fuel-python) → Fuel Library Team (fuel-library)
Changed in fuel:
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-web (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/141449

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to fuel-web (stable/6.0)

Related fix proposed to branch: stable/6.0
Review: https://review.openstack.org/141450

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-web (stable/6.0)

Change abandoned by Evgeniy L (<email address hidden>) on branch: stable/6.0
Review: https://review.openstack.org/141450
Reason: Does not work properly

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/141449
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=1dc4d6780d39b760feb16b3ea37664178db8587b
Submitter: Jenkins
Branch: master

commit 1dc4d6780d39b760feb16b3ea37664178db8587b
Author: Evgeniy L <email address hidden>
Date: Fri Dec 12 21:04:13 2014 +0400

    Urgent fix for log page

    remote_openstack_log_format should also
    include remote_syslog_log_format.

    The reason for that is in 6.0 release
    nova sends logs in new format, but we
    have to support old environments with
    old logging format.

    Change-Id: I181d08d8d482892f45f445ce887021225d246bde
    Related-bug: #1401852

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to fuel-web (stable/6.0)

Reviewed: https://review.openstack.org/141450
Committed: https://git.openstack.org/cgit/stackforge/fuel-web/commit/?id=37071fd821873e42a316997aaefa05918c957132
Submitter: Jenkins
Branch: stable/6.0

commit 37071fd821873e42a316997aaefa05918c957132
Author: Evgeniy L <email address hidden>
Date: Fri Dec 12 21:04:13 2014 +0400

    Urgent fix for log page

    remote_openstack_log_format should also
    include remote_syslog_log_format.

    The reason for that is in 6.0 release
    nova sends logs in new format, but we
    have to support old environments with
    old logging format.

    Change-Id: I181d08d8d482892f45f445ce887021225d246bde
    Related-bug: #1401852

Revision history for this message
Meg McRoberts (dreidellhasa) wrote :

I will add some information about this to http://docs.mirantis.com/openstack/fuel/fuel-5.1/operations.html#logs-and-messages -- is it okay to mention Logstash? No details about configuring it or anything, just a mention that it is useful.

Let me slog through this discussion and see if I can come up with some meaningful information. Time is short -- if anyone wants to contribute anything, it's welcome.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-docs (master)

Fix proposed to branch: master
Review: https://review.openstack.org/143312

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-docs (master)

Change abandoned by Meg McRoberts (<email address hidden>) on branch: master
Review: https://review.openstack.org/143312
Reason: Handled in Release Notes

Revision history for this message
Anastasia Palkina (apalkina) wrote :

Verified on ISO #75

"build_id": "2015-01-22_22-54-45", "ostf_sha": "95054878eff472eb6a0d9d185ce65e3105dde632", "build_number": "75", "auth_required": true, "api": "1.0", "nailgun_sha": "b677b29899f9e49532a959948d0ba68a83e41536", "production": "docker", "fuelmain_sha": "", "astute_sha": "f640780d1c7ebe89e4f343736500945525799ee0", "feature_groups": ["mirantis"], "release": "6.1", "release_versions": {"2014.2-6.1": {"VERSION": {"build_id": "2015-01-22_22-54-45", "ostf_sha": "95054878eff472eb6a0d9d185ce65e3105dde632", "build_number": "75", "api": "1.0", "nailgun_sha": "b677b29899f9e49532a959948d0ba68a83e41536", "production": "docker", "fuelmain_sha": "", "astute_sha": "f640780d1c7ebe89e4f343736500945525799ee0", "feature_groups": ["mirantis"], "release": "6.1", "fuellib_sha": "673808c5bd6ff5560df75b74f7f798649d6a0040"}}}, "fuellib_sha": "673808c5bd6ff5560df75b74f7f798649d6a0040"

tags: added: system-test-not-required
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.