Ubuntu16.04: Scons failures in analytics

[amudhar]

Ubuntu16.04: Scons failures in analytics

Below are the issues and possible patches.

Failure
=======
In file included from tools/sandesh/library/cpp/sandesh_connection.cc:14:0:
build/include/sandesh/protocol/TXMLProtocol.h: In static member function ‘static void contrail::sandesh::protocol::TXMLProtocol::unescapeXMLControlChars(std::__cxx11::string&)’:
build/include/sandesh/protocol/TXMLProtocol.h:77:7: error: ‘replace_all’ is not a member of ‘boost’
       boost::replace_all(str, "&", "&");
       ^
build/include/sandesh/protocol/TXMLProtocol.h:78:7: error: ‘replace_all’ is not a member of ‘boost’
       boost::replace_all(str, "'", "\'");
       ^
build/include/sandesh/protocol/TXMLProtocol.h:79:7: error: ‘replace_all’ is not a member of ‘boost’
       boost::replace_all(str, "&lt;", "<");
       ^
build/include/sandesh/protocol/TXMLProtocol.h:80:7: error: ‘replace_all’ is not a member of ‘boost’
       boost::replace_all(str, "&gt;", ">");
       ^
scons: *** [build/debug/tools/sandesh/library/cpp/sandesh_connection.o] Error 1
scons: building terminated because of errors.
debian/rules:28: recipe for target 'override_dh_auto_build' failed
make[2]: Leaving directory '/home/amudha/mitaka-mainline-sb/build/packages/contrail'
make[2]: *** [override_dh_auto_build] Error 2

diff
----

diff --git a/library/cpp/protocol/TXMLProtocol.h b/library/cpp/protocol/TXMLProtocol.h
index b4826e1..1f4910c 100644
--- a/library/cpp/protocol/TXMLProtocol.h
+++ b/library/cpp/protocol/TXMLProtocol.h
@@ -10,6 +10,7 @@

 #include <boost/shared_ptr.hpp>
 #include <boost/tokenizer.hpp>
+#include <boost/algorithm/string/replace.hpp>

 namespace contrail { namespace sandesh { namespace protocol {

@@ -74,10 +75,10 @@ class TXMLProtocol : public TVirtualProtocol<TXMLProtocol> {
   }

   static void unescapeXMLControlChars(std::string& str) {
- boost::replace_all(str, "&amp;", "&");
- boost::replace_all(str, "&apos;", "\'");
- boost::replace_all(str, "&lt;", "<");
- boost::replace_all(str, "&gt;", ">");
+ boost::algorithm::replace_all(str, "&amp;", "&");
+ boost::algorithm::replace_all(str, "&apos;", "\'");
+ boost::algorithm::replace_all(str, "&lt;", "<");
+ boost::algorithm::replace_all(str, "&gt;", ">");
   }
======================================================================================

Failure
=======

In file included from build/include/sandesh/sandesh_types.h:15:0,
                 from build/debug/vnsw/agent/uve/vrouter_types.cpp:12:
build/include/sandesh/derived_stats_algo.h: In instantiation of ‘contrail::sandesh::DSSum<ElemT, SumResT>::DSSum(const string&) [with ElemT = long unsigned int; SumResT = long unsigned int; std::__cxx11::string = std::__cxx11::basic_string<char>]’:
/usr/include/boost/smart_ptr/make_shared_object.hpp:808:5: required from ‘typename boost::detail::sp_if_not_array<T>::type boost::make_shared(const A1&) [with T = contrail::sandesh::DSSum<long unsigned int, long unsigned int>; A1 = std::__cxx11::basic_string<char>; typename boost::detail::sp_if_not_array<T>::type = boost::shared_ptr<contrail::sandesh::DSSum<long unsigned int, long unsigned int> >]’
build/include/sandesh/derived_stats.h:118:70: required from ‘void contrail::sandesh::DerivedStatsMerge(const std::map<std::__cxx11::basic_string<char>, ElemT>&, std::map<std::__cxx11::basic_string<char>, boost::shared_ptr<DSTT<ElemT, ResultT> > >&, std::__cxx11::string, const std::map<std::__cxx11::basic_string<char>, bool>&, uint64_t) [with DSTT = contrail::sandesh::DSSum; ElemT = long unsigned int; ResultT = long unsigned int; std::__cxx11::string = std::__cxx11::basic_string<char>; uint64_t = long unsigned int]’
build/include/sandesh/derived_stats.h:205:50: required from ‘void contrail::sandesh::DerivedStatsIf<DSTT, ElemT, ResultT>::Update(const std::map<std::__cxx11::basic_string<char>, ElemT>&, const std::map<std::__cxx11::basic_string<char>, bool>&, uint64_t) [with DSTT = contrail::sandesh::DSSum; ElemT = long unsigned int; ResultT = long unsigned int; uint64_t = long unsigned int]’
build/debug/vnsw/agent/uve/vrouter_types.cpp:12287:102: required from here
build/include/sandesh/derived_stats_algo.h:277:42: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
             while ((1 << (shifter_ + 8)) < range_usecs_) shifter_++;
                                          ^
cc1plus: all warnings being treated as errors
scons: *** [build/debug/vnsw/agent/uve/vrouter_types.o] Error 1
scons: building terminated because of errors.
debian/rules:28: recipe for target 'override_dh_auto_build' failed

diff
----
diff --git a/library/cpp/derived_stats_algo.h b/library/cpp/derived_stats_algo.h
index 347b801..79e3866 100644
--- a/library/cpp/derived_stats_algo.h
+++ b/library/cpp/derived_stats_algo.h
@@ -274,7 +274,7 @@ class DSSum {
             // We want each time bucket to represent between
             // 1/128th and 1/256th of the entire range
             // This ensures that there will never be more than 256 buckets
- while ((1 << (shifter_ + 8)) < range_usecs_) shifter_++;
+ while ((uint64_t)(1 << (shifter_ + 8)) < range_usecs_) shifter_++;
         }
     }
     uint64_t samples_;
======================================================================================

Failure
=======
controller/src/discovery/client -Ibuild/debug/analytics -Icontroller/src/analytics -Ibuild/debug -Ibuild/debug/base/sandesh -Icontroller/src/base/sandesh controller/src/analytics/syslog_collector.cc
controller/src/analytics/syslog_collector.cc:19:58: fatal error: boost/spirit/home/phoenix/object/construct.hpp: No such file or directory
compilation terminated.
scons: *** [build/debug/analytics/syslog_collector.o] Error 1
scons: building terminated because of errors.
debian/rules:28: recipe for target 'override_dh_auto_build' failed
make[2]: Leaving directory '/home/amudha/mitaka-mainline-sb/build/packages/contrail'
make[2]: *** [override_dh_auto_build] Error 2
debian/rules:25: recipe for target 'build' failed
make[1]: Leaving directory '/home/amudha/mitaka-mainline-sb/build/packages/contrail'
make[1]: *** [build] Error 2

diff
----
diff --git a/src/analytics/syslog_collector.cc b/src/analytics/syslog_collector.cc
index 643917c..3e778b5 100644
--- a/src/analytics/syslog_collector.cc
+++ b/src/analytics/syslog_collector.cc
@@ -16,7 +16,7 @@
 #include <boost/spirit/include/phoenix_core.hpp>
 #include <boost/spirit/include/phoenix_operator.hpp>
 #include <boost/spirit/include/phoenix_stl.hpp>
-#include <boost/spirit/home/phoenix/object/construct.hpp>
+#include <boost/phoenix/object/construct.hpp>
 #include <boost/uuid/uuid.hpp>
 #if __GNUC_PREREQ(4, 6)
 #pragma GCC diagnostic push
======================================================================================

Failure
=======
Install file: "build/third_party/hiredis/libhiredis.a" as "build/lib/libhiredis.a"
g++ -o build/debug/zookeeper/zookeeper_client.o -c -g -O0 -DDEBUG -Wall -Werror -Wsign-compare -Wno-unused-local-typedefs -DLINUX -Icontroller/src -Ibuild/include -Icontroller/lib -I/usr/include controller/src/zookeeper/zookeeper_client.cc
controller/src/zookeeper/zookeeper_client.cc: In member function ‘int zookeeper::client::impl::ZookeeperClientImpl::CreateNodeSync(const char*, const char*, int*)’:
controller/src/zookeeper/zookeeper_client.cc:167:67: error: ‘strlen’ was not declared in this scope
         rc = zki_->ZooCreate(zk_handle_, path, value, strlen(value),
                                                                   ^
scons: *** [build/debug/zookeeper/zookeeper_client.o] Error 1

diff
----
diff --git a/src/zookeeper/zookeeper_client.cc b/src/zookeeper/zookeeper_client.cc
index c3c5ff8..3e380cc 100644
--- a/src/zookeeper/zookeeper_client.cc
+++ b/src/zookeeper/zookeeper_client.cc
@@ -3,6 +3,7 @@
 //

 #include <cerrno>
+#include <cstring>

 #include <base/logging.h>
=====================================================================================

Failure
=======
scons: `build/debug/vnsw/agent/contrail/contrail-vrouter-agent' is up to date.
g++ -o build/debug/analytics/vizd -Wl,--whole-archive -lbase -lcpuinfo -lprocess_info -lnodeinfo -l:librdkafka.a -l:librdkafka++.a -Wl,--no-whole-archive -g build/debug/analytics/collector_uve_types.o build/debug/analytics/collector_uve_constants.o build/debug/analytics/collector_uve_html.o build/debug/analytics/analytics_types.o build/debug/analytics/analytics_constants.o build/debug/analytics/analytics_html.o build/debug/analytics/redis_types.o build/debug/analytics/redis_constants.o build/debug/analytics/redis_html.o build/debug/analytics/uflow_types.o build/debug/analytics/uflow_constants.o build/debug/analytics/uflow_html.o build/debug/analytics/sflow_types.o build/debug/analytics/sflow_constants.o build/debug/analytics/sflow_html.o build/debug/analytics/viz_types.o build/debug/analytics/viz_constants.o build/debug/analytics/viz_html.o build/debug/analytics/viz_collector.o build/debug/analytics/ruleeng.o build/debug/analytics/collector.o build/debug/analytics/vizd_table_desc.o build/debug/analytics/viz_message.o build/debug/analytics/generator.o build/debug/analytics/redis_connection.o build/debug/analytics/redis_processor_vizd.o build/debug/analytics/options.o build/debug/analytics/stat_walker.o build/debug/analytics/protobuf_collector.o build/debug/analytics/protobuf_server.o build/debug/analytics/sflow.o build/debug/analytics/sflow_generator.o build/debug/analytics/sflow_collector.o build/debug/analytics/usrdef_counters.o build/debug/analytics/sflow_parser.o build/debug/analytics/ipfix_collector.o build/debug/analytics/protobuf_schema.pb.o build/debug/analytics/main.o build/debug/analytics/buildinfo.o build/debug/analytics/db_handler.o build/debug/analytics/OpServerProxy.o build/debug/analytics/syslog_collector.o build/debug/analytics/parser_util.o -Lbuild/lib -Lbuild/debug/sandesh -Lcontroller/src/sandesh -Lbuild/debug/analytics/ruleparser -Lcontroller/src/analytics/ruleparser -Lbuild/debug/discovery/client -Lcontroller/src/discovery/client -Lbuild/debug/http/client -Lcontroller/src/http/client -Lbuild/debug/xml -Lcontroller/src/xml -Lbuild/debug/database -Lcontroller/src/database -Lbuild/debug/base -Lcontroller/src/base -Lbuild/debug/io -Lcontroller/src/io -Lbuild/debug/net -Lcontroller/src/net -L/usr/lib/x86_64-linux-gnu -lcassandra_cql -lcassandra -lgendb -lboost_regex -lssl -lcrypto -lsasl2 -lcpuinfo -lds -lio -lsandesh -lhttpc -lhttp -lvncapi -lhttp_parser -lcurl -lruleparser -lxml -lprocess_info -lnodeinfo -lbase -lio -lz -lnet -lipfix -lmisc -lsandeshvns -lpugixml -lhiredis -lprotobuf -lzookeeper_client -lzookeeper_mt -lboost_filesystem -lboost_program_options -lboost_system -llog4cplus -lpthread -ltbb -lboost_python -lpython2.7 -lrt
/usr/bin/ld: cannot find -lsasl2
collect2: error: ld returned 1 exit status

diff
----
(PLATFORM, VERSION, EXTRA) = platform.linux_distribution()
if PLATFORM.lower() != 'ubuntu':
    AnalyticsEnv.Prepend(LIBS=['ssl',
        'crypto',
        'sasl2'])

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/25896
Submitter: Sundaresan Rajangam (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/25907
Submitter: Sundaresan Rajangam (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/25896
Committed: http://github.org/Juniper/contrail-sandesh/commit/8b6c1388e9574ab971952734c71d0a5f6ecb8280
Submitter: Zuul
Branch: master

commit 8b6c1388e9574ab971952734c71d0a5f6ecb8280
Author: Sundaresan Rajangam <email address hidden>
Date: Wed Nov 9 12:26:40 2016 -0800

Fix sandesh compilation error in ubuntu 16.04

Change-Id: Iacbbf4c6e38cc4168e4c8e7dd7df3344fe638866
Closes-Bug: #1638636

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/25907
Committed: http://github.org/Juniper/contrail-controller/commit/7ef887e340485414b9c2c1be8b44cda74dd0fcf3
Submitter: Zuul
Branch: master

commit 7ef887e340485414b9c2c1be8b44cda74dd0fcf3
Author: Sundaresan Rajangam <email address hidden>
Date: Wed Nov 9 16:11:39 2016 -0800

Fix analytics compilation failure in ubuntu 16.04

Change-Id: Ie07529fde0b26938cfcadcad5dc1f84b958744cb
Closes-Bug: #1638636

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/36457
Submitter: Antoine Eiche (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/36457
Committed: http://github.com/Juniper/contrail-sandesh/commit/fdc9aec2e599b5bc93699a90c97952f417dc7916
Submitter: Zuul (<email address hidden>)
Branch: R3.2

commit fdc9aec2e599b5bc93699a90c97952f417dc7916
Author: Sundaresan Rajangam <email address hidden>
Date: Wed Nov 9 12:26:40 2016 -0800

Fix sandesh compilation error in ubuntu 16.04

Change-Id: Iacbbf4c6e38cc4168e4c8e7dd7df3344fe638866
Closes-Bug: #1638636
(cherry picked from commit 8b6c1388e9574ab971952734c71d0a5f6ecb8280)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/48817
Submitter: Vinay Vithal Mahuli (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/48851
Submitter: Vinay Vithal Mahuli (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/48851
Committed: http://github.com/Juniper/contrail-sandesh/commit/7489c5eab9c76311abe356d4babdbcade6055067
Submitter: Zuul (<email address hidden>)
Branch: R3.1

commit 7489c5eab9c76311abe356d4babdbcade6055067
Author: Sundaresan Rajangam <email address hidden>
Date: Wed Nov 9 12:26:40 2016 -0800

Fix sandesh compilation error in ubuntu 16.04

Closes-Bug: #1638636

Fixing ubuntu1604 build break due to tbb mutex not being copyable or clonable.
Closes-Bug: 1670880

Remove conflict with linux uuid_t type
Partial-Bug: 1633387

Optimize sandesh C generated code
Remove thrift_protocol and thrift_transport layers and directly
copy to/from buffer and structures in generated code for read
and write. This results in 10 times improvement in binary encode
and decode.
Closes-Bug: 1604700

Change-Id: Id2ee27b8d8b8ad2c0b6b31245eeb6b92ce7e4a16

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/49348
Submitter: Vinay Vithal Mahuli (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/49348
Committed: http://github.com/Juniper/contrail-controller/commit/50bd186e5e07c7904805bad89a41a0518004930c
Submitter: Zuul (<email address hidden>)
Branch: R3.1

commit 50bd186e5e07c7904805bad89a41a0518004930c
Author: Prakash Bailkeri <email address hidden>
Date: Mon Nov 7 10:10:41 2016 +0530

Fix compilation error for ubuntu 16.04

Include iostream explicitly in ifmap_node.cc as it uses std::cout.

Earlier this include file was coming via boost/assert.h. In the version of boost
used in 16.04 this file(boost/assert.h) has been modified.

Closes-Bug: #1638644

Fix build with newer TBB

Partial-Bug: #1603881

Ubuntu1604: build and packaging changes

Setting right path for gtest

Partial-Bug: #1638584

Fix analytics compilation failure in ubuntu 16.04

Change-Id: I0c0d1f73aaa136398f992b32b43f7ce2b6218a88
Closes-Bug: #1638636