liberasure produces unnecessary messages in user.log

[Daniil Lapshin]

Detailed bug description:
 If swift is installed liberasure produces the following messages every 5 minutes that end up being written to /var/log/user.log:

 <11>Dec 6 16:30:02 node-6 liberasurecode[29424]: liberasurecode_backend_open: dynamic linking error libJerasure.so.2: cannot open shared object file: No such file or directory
 <11>Dec 6 16:30:02 node-6 liberasurecode[29424]: liberasurecode_backend_open: dynamic linking error libJerasure.so.2: cannot open shared object file: No such file or directory
 <11>Dec 6 16:30:02 node-6 liberasurecode[29424]: liberasurecode_backend_open: dynamic linking error libisal.so.2: cannot open shared object file: No such file or directory
 <11>Dec 6 16:30:02 node-6 liberasurecode[29424]: liberasurecode_backend_open: dynamic linking error libshss.so.1: cannot open shared object file: No such file or directory

 As I understand messages generated because on liberasure pluggable architecture which tries to use all backend plugins even if they aren't installed.

 This issue looks cosmetic but nevertheless log files filled with unnecessary messages could be quite disturbing.

Steps to reproduce:
 Deploy MOS with Swift

Expected results:
 liberasure produces only relevant messages

Actual result:
 liberasure complain about missing shared objects every 5 minutes

Reproducibility:
 All the time

Workaround:
 Syslog could be configured to write liberasure logs in separate file (or ignore them)

Impact:
 Cosmetic

Description of the environment:
- Operation system: MOS 9.0, Ubuntu 14.04
- Related projects installed: Swift

[Scott Machtmes (smachtmes)]

I have another customer case with similar issue. This appears to have been fixed in the pyeclib code as follows: https://bitbucket.org/kmgreen2/pyeclib/commits/4f278195a42ea0f254114c92043b6ba6ecdc6b0a

[Denis Meltsaykin (dmeltsaykin)]

It seems that version https://github.com/openstack/pyeclib/tree/v1.2.1 contains a fix for this issue. We can update our fork.

[Alexey Stupnikov (astupnikov)]

I have doublechecked the changes made in python-pyeclib-1.2.1 and it looks like we will not be able to backport whole package: it requires liberasurecode-dev (>= 1.2.0), which is available only in U16.10. I will backport the fix only.

[Fuel Devops McRobotson (fuel-devops-robot)]

Fix proposed to branch: 9.0
Change author: Alexey Stupnikov <email address hidden>
Review: https://review.fuel-infra.org/36070

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/36070
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0

Commit: d5ecf646e6996a988ca347459fb053a9cf2f59c9
Author: Alexey Stupnikov <email address hidden>
Date: Thu Jul 27 12:30:50 2017

Use the check_backend_available function.

It turns out that we are unable to backport python-pyeclib v1.2.1,
since it requires liberasurecode-dev from ubuntu 16.10. I have
decided to cherry-pick only the patch we need to fix the bug.

This patch eliminates the spurrious syslog messages and is a
cleaner mechanism for querying all of the available backends on a
system.

Change-Id: I76d0f27dc35da1c68132ef39c623da9cbaa819fa
Closes-bug: #1647797

[Aleksey Zvyagintsev (azvyagintsev)]

Those merge broke swift:
http://paste.openstack.org/show/617554/

root@node-4:/var/log/upstart# dpkg -l |grep -i "swift\|glacne"
ii python-swift 2.7.1-4~u14.04+mos2 all distributed virtual object store - Python libraries
ii python-swiftclient 1:3.0.0-2~u14.04+mos1 all Client library for Openstack Swift API - Python 2.7
ii swift 2.7.1-4~u14.04+mos2 all distributed virtual object store - common files
ii swift-account 2.7.1-4~u14.04+mos2 all distributed virtual object store - account server
ii swift-container 2.7.1-4~u14.04+mos2 all distributed virtual object store - container server
ii swift-object 2.7.1-4~u14.04+mos2 all distributed virtual object store - object server
ii swift-plugin-s3 1.9-1~u14.04+mos1 all swift3 (S3 compatibility) middleware plugin for swift
ii swift-proxy 2.7.1-4~u14.04+mos2 all distributed virtual object store - proxy server

[Aleksey Zvyagintsev (azvyagintsev)]

dpkg -l |grep -i "python-pyeclib"
ii python-pyeclib 1.2.0-2~u14.04+mos2 amd64 interface for implementing erasure codes - Python 2.x

[Alexey Stupnikov (astupnikov)]

This is a known bug in pyeclib v1.2.1. Need to backport a fix for bug #1586220

[Alexey Stupnikov (astupnikov)]

OK, here is the situation: the patch I have backported requires missing function. Need to discuss the situation and possible solutions.

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: 9.0
Change author: Alexey Stupnikov <email address hidden>
Review: https://review.fuel-infra.org/36176

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/36176
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0

Commit: 2a7f7be1adaccb355f4f92f37df6977cbcd32dd7
Author: Alexey Stupnikov <email address hidden>
Date: Tue Aug 8 15:48:04 2017

Revert "Use the check_backend_available function."

To make this patch fully functional we need to backport two
patches to openstack/liberasurecode (c7a94df and fd88e70).

This will break backward compatibility between python-pyeclib
and liberasurecode1 packages, so we need to bump major package
versions, then change requirements. Our version will not match
upstream one and it will become a one large mess.

IMO we should just backport this fix for Medium bug and
don't touch a functional code.

This reverts commit d5ecf646e6996a988ca347459fb053a9cf2f59c9.

Change-Id: I7fb21a141be4de6b1c82b5b78216ac488fc82d61
Related-bug: #1647797

[Alexey Stupnikov (astupnikov)]

The fix for this issue is too complex and error-prone. We don't merge such fixes, moving to Won't Fix.

[Denis Meltsaykin (dmeltsaykin)]

I'm also downgrading the importance to medium, since this issue is obviously cosmetic rather than a real game stopper.