After upgrade to 4.0.1 and migration to graphite, we have problem to work with netmap and stats.

Bug #1304922 reported by limo
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Network Administration Visualized
Invalid
Undecided
Unassigned

Bug Description

. We have maybe unusual configuration. Nav is runing on https://hostname:8080/
I enabled carbon cache and udp, and included nav/apache.conf to SSL virtualhost on port 8080. Next, I updated base to https://server:8080/ in graphite.conf. But when I want to see stats, I see

JSONDecodeError: No JSON object could be decoded: line 2 column 1 (char 1)
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 111, in get_response response = callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python2.7/dist-packages/nav/web/sortedstats/views.py", line 37, in index result.collect()
File "/usr/lib/python2.7/dist-packages/nav/web/sortedstats/statmodules.py", line 59, in collect self.data = self.get_sorted_data()
File "/usr/lib/python2.7/dist-packages/nav/web/sortedstats/statmodules.py", line 67, in get_sorted_data data = self.get_data()
File "/usr/lib/python2.7/dist-packages/nav/web/sortedstats/statmodules.py", line 74, in get_data data = get_metric_average(target, start=self.timeframe)
File "/usr/lib/python2.7/dist-packages/nav/metrics/data.py", line 38, in get_metric_average data = get_metric_data(target, start, end)
File "/usr/lib/python2.7/dist-packages/nav/metrics/data.py", line 95, in get_metric_data return simplejson.load(response)
File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 395, in load use_decimal=use_decimal, **kw)
File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 451, in loads return _default_decoder.decode(s)
File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 402, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 420, in raw_decode raise JSONDecodeError("No JSON object could be decoded", s, idx)
JSONDecodeError: No JSON object could be decoded: line 2 column 1 (char 1)

Revision history for this message
Morten Brekkevold (mbrekkevold) wrote :

Let me make sure I understand you correctly: https://hostname:8080/ and https://server:8080/ are in fact two separate SSL-enabled servers?

The error would mean that NAV cannot get a proper response from the API at https://server:8080/ - I cannot reproduce this by switching on SSL on a Graphite installation, so I would need more details.

Can you set DJANGO_DEBUG=True in `nav.conf` and restart apache, then load the crashing page again? I would really like to see the contents of the expandable local vars on this line:

File "/usr/lib/python2.7/dist-packages/nav/metrics/data.py", line 95, in get_metric_data
  return simplejson.load(response)

Changed in nav:
status: New → Incomplete
Revision history for this message
limo (t-lukas) wrote :

Sorry, I already repaired graphs. We share nav installation with zabbix and there was mod_pagespeed enabled in apache. I disabled it and netmap is working again.

Variables from line 95:
Variable Value
end
'now'
target
'highestAverage(group(nav.devices.*.cpu.*.loadavg5min,nav.devices.*.cpu.cpu.utilization), 5)'
url
'https://zabbix.opf.slu.cz:8080/render/'
req
<urllib2.Request instance at 0x7fdf66396908>
start
u'-1d'
base
'https://zabbix.opf.slu.cz:8080/'
query
'format=json&from=-1d&target=highestAverage%28group%28nav.devices.%2A.cpu.%2A.loadavg5min%2Cnav.devices.%2A.cpu.cpu.utilization%29%2C+5%29&until=now'
response
<addinfourl at 140597469473680 whose fp = None>

May it be that we do not use cricket? We use nav only for basic monitoring of SNMP devices. For graphs and history of data, we use Zabbix.
Thank you

Revision history for this message
Morten Brekkevold (mbrekkevold) wrote :

Ah, yes, using page-modifying modules like mod_pagespeed for serving REST APIs over Apache would break things.

Changed in nav:
status: Incomplete → Invalid
Revision history for this message
limo (t-lukas) wrote :

But it seems that this solved only one problem, with maps. It works now. But lot of pages are unreachable now.
Please can you explain, how that graphite and django api should work? There is probably some misconfiguration in apache.
I inspected error log and try to look into return simplejson.load(response)
where url in local vars is 'https://zabbix.opf.slu.cz:8080/metrics/find?query=nav.devices.swc4-serverovna_opf_slu_cz.%2A'
I think that there should be some JSON data on that url ?

If I try to open that url, I see
Page not found (404)
Request Method: GET
Request URL: https://zabbix.opf.slu.cz:8080/metrics/find?query=nav.devices.swc4-serverovna_opf_slu_cz.%2A
Using the URLconf defined in nav.django.urls, Django tried these URL patterns, in this order:
^$ [name='webfront-index']
^index/login/ [name='webfront-login']
^index/logout/ [name='webfront-logout']
^about/ [name='webfront-about']
^toolbox/$ [name='webfront-toolbox']
^toolbox/savetools [name='webfront-save-toolbox']
^toolbox/changelayout [name='webfront-set-tool-layout']
^preferences/$ [name='webfront-preferences']
^preferences/savelinks$ [name='webfront-preferences-savelinks']
^preferences/changepassword$ [name='webfront-preferences-changepassword']
^status/
^ajax/
^alertprofiles/
^api/
^arnold/
^devicehistory/
^geomap/
^search/
^ipdevinfo/
^l2trace/
^machinetracker/
^macwatch/
^maintenance/
^messages/
^netmap/
^networkexplorer/
^portadmin/
^powersupplies/
^radius/
^report/
^seeddb/
^stats/
^syslogger/
^threshold/
^graphite/
^navlets/
^useradmin/
^static\/(?P<path>.*)$

The current URL, metrics/find, didn't match any of these.
/metrics should be probably somewhere else?

Thank you very much

Revision history for this message
Morten Brekkevold (mbrekkevold) wrote :

https://zabbix.opf.slu.cz:8080/metrics/find?query=nav.devices.swc4-serverovna_opf_slu_cz.%2A is a request to Graphite, not to NAV.

You are serving NAV at zabbix.opf.slu.cz:8080, so you have a configuration problem. NAV's graphite.conf must point to Graphite's base URL, not NAV's own base URL.

Revision history for this message
limo (t-lukas) wrote :

Thank you very much, now I understand and repaired it manualy. Nav upgrade script was not successful probably because of nonstandard web settings.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.