database-nodemgr doesn't get number correctly

[Tony Liu]

database_nodemgr/database_event_manager.py

========
    popen_cmd = "set `df -Pk " + analytics_dir + " | grep % | awk '{s+=$4}END{print s}'` && echo $1"
========

# df -Pk /var/lib/cassandra/data/ContrailAnalyticsCql
Filesystem 1024-blocks Used Available Capacity Mounted on
/dev/mapper/5b4s8--vg-root 7109263368 10196708 6740757592 1% /

# df -Pk /var/lib/cassandra/data/ContrailAnalyticsCql | grep % | awk '{s+=$4}END{print s}'
6.74076e+09

I fixed it with this patch.
========
- popen_cmd = "set `df -Pk " + analytics_dir + " | grep % | awk '{s+=$4}END{print s}'` && echo $1"
+ popen_cmd = "set `df -Pk " + analytics_dir + " | grep % | awk '{print $4}'` && echo $1"
========

The same issue may happens to other fields of df output.

[Raj Reddy (rajreddy)]

Tony, which OS is this, I don't see this on ubuntu 14 or cento 7.1

In any case can you see if change
{print s}
to
{printf "%i", s}
works.

[Tony Liu (taoliu-7)]

The server where we see this issue is 5b4s8 10.87.64.132.
You can login to see why this happens there.

root@5b4s8:~# uname -a
Linux 5b4s8 3.13.0-40-generic #69-Ubuntu SMP Thu Nov 13 17:53:56 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
root@5b4s8:~# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.2 LTS"
root@5b4s8:~# echo "/dev/mapper/5b4s8--vg-root 7109263368 10196708 6740757592 1% /" | awk '{s+=$4}END{print s}'
6.74076e+09
root@5b4s8:~# echo "/dev/mapper/5b4s8--vg-root 7109263368 10196708 6740757592 1% /" | awk '{s+=$4}END{printf "%i", s}'
2147483647

Your suggestion works.

[Raj Reddy (rajreddy)]

where the problem is reported, there's mawk is installed and it's getting called for awk..

root@5b4s8:~# ls -l /usr/bin/awk
lrwxrwxrwx 1 root root 21 Mar 1 17:35 /usr/bin/awk -> /etc/alternatives/awk
root@5b4s8:~# ls -l /etc/alternatives/awk
lrwxrwxrwx 1 root root 13 Mar 1 17:35 /etc/alternatives/awk -> /usr/bin/mawk
root@5b4s8:~# dpkg -S /usr/bin/mawk
mawk: /usr/bin/mawk

Where as in our regular machines it's gawk -- hence the difference..

Regardless,
. we can use popen call as in

import subprocess
df = subprocess.Popen(["df", "filename"], stdout=subprocess.PIPE)
output = df.communicate()[0]
device, size, used, available, percent, mountpoint = \
    output.split("\n")[1].split()

OR

use os.statvfs()..

import os
statvfs = os.statvfs('/home/foo/bar/baz')

statvfs.f_frsize * statvfs.f_blocks # Size of filesystem in bytes
statvfs.f_frsize * statvfs.f_bfree # Actual number of free bytes

------
There's additional issue that data_file_directories can be multiple and we should be summ'ing up the space on all of them..

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/18203
Submitter: Nikhil Bansal (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/18203
Committed: http://github.org/Juniper/contrail-controller/commit/e68c499aaf89a2eb404abcf700828a8bb4274685
Submitter: Zuul
Branch: master

commit e68c499aaf89a2eb404abcf700828a8bb4274685
Author: Nikhil B <email address hidden>
Date: Mon Mar 7 10:04:38 2016 +0530

Fixing the parse issue due to mawk

In some setups, if mawk is installed, awk does not output numbers correctly.
With this patch, awk is being avoided altogether to alleviate such issues

Change-Id: I14befc9a72202bfe7c602b004f7ac6972a8efc60
Closes-Bug: 1552477

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/18421
Submitter: Nikhil Bansal (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/18421
Committed: http://github.org/Juniper/contrail-controller/commit/7b6ce73ff5a7e6b287b73fe904ecdb95e3f2191c
Submitter: Zuul
Branch: R3.0

commit 7b6ce73ff5a7e6b287b73fe904ecdb95e3f2191c
Author: Nikhil B <email address hidden>
Date: Mon Mar 7 10:04:38 2016 +0530

Fixing the parse issue due to mawk

In some setups, if mawk is installed, awk does not output numbers correctly.
With this patch, awk is being avoided altogether to alleviate such issues

Change-Id: I14befc9a72202bfe7c602b004f7ac6972a8efc60
Closes-Bug: 1552477
(cherry picked from commit e68c499aaf89a2eb404abcf700828a8bb4274685)