diff -Nru rsyslog-8.12.0/ChangeLog rsyslog-8.14.0/ChangeLog --- rsyslog-8.12.0/ChangeLog 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/ChangeLog 2015-11-03 08:34:15.000000000 +0100 @@ -1,4 +1,64 @@ ------------------------------------------------------------------------------ +Version 8.14.0 [v8-stable] 2015-11-03 +- add property "rawmsg-after-pri" +- bugfix: potential misadresseing in imfile + Could happen when wildcards were used. + see also https://github.com/rsyslog/rsyslog/issues/532 + see also https://github.com/rsyslog/rsyslog/issues/534 + Thanks to zhangdaoling for the bugfix. +- bugfix: re_extract RainerScript function did not work + Thanks to Janmejay Singh for the patch +------------------------------------------------------------------------------ +Version 8.13.0 [v8-stable] 2015-09-22 +- ZeroMQ enhancements: + * Added the ability to set a static publishing topic per action as an + alternative to constructing topics with templates + Contributor: Luca Bocassi + * ZMQ_PUB socket now defaults to bind and ZMQ_SUB socket now defaults to + connect - Contributor: Luca Bocassi +- Redis enhancements: + * Can now LPUSH to a Redis list in "queue" mode - Contributor: Brian Knox + * Can now PUBLISH to a Redis channel in "publish" mode + Contributor: Brian Knox +- build requirement for rsyslog/mmnormalize is now liblognorm 1.1.2 or above +- mmnormalize: liblognorm error messages are now emitted via regular + rsyslog error reporting mechanism (aka "are now logged") + This is possible due to a new API in liblognorm 1.1.2; + Note that the amount of error messages depends on the version of + liblognorm used. +- add support for TCP client side keep-alives + Thanks to github user tinselcity for the patch. +- bugfix: imtcp/TLS hangs on dropped packets + see also https://github.com/rsyslog/rsyslog/issues/318 + Thanks to github user tinselcity for the patch. +- bugfix testbench: some tests using imptcp are run if module is disabled + Thanks to Michael Biebl for reporting this + see also https://github.com/rsyslog/rsyslog/issues/524 +- bugfix omkafka: Fixes a bug not accepting new messages anymore. + see also: https://github.com/rsyslog/rsyslog/pull/472 + Thanks to Janmejay Singh +- bugfix: Parallel build issue "cannot find ../runtime/.libs/librsyslog.a: + No such file or directory" (#479) fixed. + Thanks to Thomas D. (Whissi) for the patch. +- bugfix: Added missing mmpstructdata testfiles into makefile. + see also: https://github.com/rsyslog/rsyslog/issues/484 +- bugfix: Reverted FIX for issue #392 as it had unexpected side effects. + The new fix duplicates the Listener object for static files (like + done for dynamic files already), resolving issue #392 and #490. + see also https://github.com/rsyslog/rsyslog/pull/490 +- bugfix: issues in queue subsystem if syncqueuefiles was enabled + * Error 14 was generated on the .qi file directory handle. + As the .qi filestream does not have a directory set, fsync + was called on an empty directory causing a error 14 in debug log. + * When queue files existed on startup, the bSyncQueueFiles + strm property was not set to 1. This is now done in the + qqueueLoadPersStrmInfoFixup function. +- bugfix/testbench: tcpflood tool could abort when random data was added + see also: https://github.com/rsyslog/rsyslog/issues/506 + Thanks to Louis Bouchard for the fix +- rscryutil: Added support to decrypt a not closed log file. + Thanks to wizard1024 for the patch. +------------------------------------------------------------------------------ Version 8.12.0 [v8-stable] 2015-08-11 - Harmonize resetConfigVariables values and defaults see also https://github.com/rsyslog/rsyslog/pull/413 diff -Nru rsyslog-8.12.0/config.h.in rsyslog-8.14.0/config.h.in --- rsyslog-8.12.0/config.h.in 2015-08-10 12:25:55.000000000 +0200 +++ rsyslog-8.14.0/config.h.in 2015-11-03 09:52:20.000000000 +0100 @@ -156,9 +156,6 @@ /* Define to 1 if you have the header file. */ #undef HAVE_LIBRDKAFKA_RDKAFKA_H -/* gnutls is available */ -#undef HAVE_LIB_GNUTLS - /* Define to 1 if you have the header file. */ #undef HAVE_LOCALE_H diff -Nru rsyslog-8.12.0/configure rsyslog-8.14.0/configure --- rsyslog-8.12.0/configure 2015-08-10 12:25:54.000000000 +0200 +++ rsyslog-8.14.0/configure 2015-11-03 09:52:07.000000000 +0100 @@ -1,6 +1,6 @@ #! /bin/sh # Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.69 for rsyslog 8.12.0. +# Generated by GNU Autoconf 2.69 for rsyslog 8.14.0. # # Report bugs to . # @@ -590,8 +590,8 @@ # Identity of this package. PACKAGE_NAME='rsyslog' PACKAGE_TARNAME='rsyslog' -PACKAGE_VERSION='8.12.0' -PACKAGE_STRING='rsyslog 8.12.0' +PACKAGE_VERSION='8.14.0' +PACKAGE_STRING='rsyslog 8.14.0' PACKAGE_BUGREPORT='rsyslog@lists.adiscon.com' PACKAGE_URL='' @@ -1656,7 +1656,7 @@ # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF -\`configure' configures rsyslog 8.12.0 to adapt to many kinds of systems. +\`configure' configures rsyslog 8.14.0 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... @@ -1726,7 +1726,7 @@ if test -n "$ac_init_help"; then case $ac_init_help in - short | recursive ) echo "Configuration of rsyslog 8.12.0:";; + short | recursive ) echo "Configuration of rsyslog 8.14.0:";; esac cat <<\_ACEOF @@ -2007,7 +2007,7 @@ test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF -rsyslog configure 8.12.0 +rsyslog configure 8.14.0 generated by GNU Autoconf 2.69 Copyright (C) 2012 Free Software Foundation, Inc. @@ -2587,7 +2587,7 @@ This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. -It was created by rsyslog $as_me 8.12.0, which was +It was created by rsyslog $as_me 8.14.0, which was generated by GNU Autoconf 2.69. Invocation command line was $ $0 $@ @@ -3450,7 +3450,7 @@ # Define the identity of the package. PACKAGE='rsyslog' - VERSION='8.12.0' + VERSION='8.14.0' cat >>confdefs.h <<_ACEOF @@ -18263,54 +18263,8 @@ $as_echo "#define ENABLE_GNUTLS 1" >>confdefs.h - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for gnutls_global_init in -lgnutls" >&5 -$as_echo_n "checking for gnutls_global_init in -lgnutls... " >&6; } -if ${ac_cv_lib_gnutls_gnutls_global_init+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_check_lib_save_LIBS=$LIBS -LIBS="-lgnutls - $LIBS" -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char gnutls_global_init (); -int -main () -{ -return gnutls_global_init (); - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - ac_cv_lib_gnutls_gnutls_global_init=yes -else - ac_cv_lib_gnutls_gnutls_global_init=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext -LIBS=$ac_check_lib_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_gnutls_gnutls_global_init" >&5 -$as_echo "$ac_cv_lib_gnutls_gnutls_global_init" >&6; } -if test "x$ac_cv_lib_gnutls_gnutls_global_init" = xyes; then : - - -$as_echo "#define HAVE_LIB_GNUTLS 1" >>confdefs.h - - -else - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: gnutls_global_init function missing or not detected" >&5 -$as_echo "$as_me: WARNING: gnutls_global_init function missing or not detected" >&2;} -fi - + save_libs=$LIBS + LIBS="$LIBS $GNUTLS_LIBS" for ac_func in gnutls_certificate_set_retrieve_function do : ac_fn_c_check_func "$LINENO" "gnutls_certificate_set_retrieve_function" "ac_cv_func_gnutls_certificate_set_retrieve_function" @@ -18333,7 +18287,9 @@ fi done + LIBS=$save_libs fi + if test x$enable_gnutls = xyes; then ENABLE_GNUTLS_TRUE= ENABLE_GNUTLS_FALSE='#' @@ -18666,12 +18622,12 @@ pkg_cv_LIBLOGNORM_CFLAGS="$LIBLOGNORM_CFLAGS" elif test -n "$PKG_CONFIG"; then if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"lognorm >= 1.0.2\""; } >&5 - ($PKG_CONFIG --exists --print-errors "lognorm >= 1.0.2") 2>&5 + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"lognorm >= 1.1.2\""; } >&5 + ($PKG_CONFIG --exists --print-errors "lognorm >= 1.1.2") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then - pkg_cv_LIBLOGNORM_CFLAGS=`$PKG_CONFIG --cflags "lognorm >= 1.0.2" 2>/dev/null` + pkg_cv_LIBLOGNORM_CFLAGS=`$PKG_CONFIG --cflags "lognorm >= 1.1.2" 2>/dev/null` test "x$?" != "x0" && pkg_failed=yes else pkg_failed=yes @@ -18683,12 +18639,12 @@ pkg_cv_LIBLOGNORM_LIBS="$LIBLOGNORM_LIBS" elif test -n "$PKG_CONFIG"; then if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"lognorm >= 1.0.2\""; } >&5 - ($PKG_CONFIG --exists --print-errors "lognorm >= 1.0.2") 2>&5 + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"lognorm >= 1.1.2\""; } >&5 + ($PKG_CONFIG --exists --print-errors "lognorm >= 1.1.2") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then - pkg_cv_LIBLOGNORM_LIBS=`$PKG_CONFIG --libs "lognorm >= 1.0.2" 2>/dev/null` + pkg_cv_LIBLOGNORM_LIBS=`$PKG_CONFIG --libs "lognorm >= 1.1.2" 2>/dev/null` test "x$?" != "x0" && pkg_failed=yes else pkg_failed=yes @@ -18709,14 +18665,14 @@ _pkg_short_errors_supported=no fi if test $_pkg_short_errors_supported = yes; then - LIBLOGNORM_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "lognorm >= 1.0.2" 2>&1` + LIBLOGNORM_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "lognorm >= 1.1.2" 2>&1` else - LIBLOGNORM_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "lognorm >= 1.0.2" 2>&1` + LIBLOGNORM_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "lognorm >= 1.1.2" 2>&1` fi # Put the nasty error message in config.log where it belongs echo "$LIBLOGNORM_PKG_ERRORS" >&5 - as_fn_error $? "Package requirements (lognorm >= 1.0.2) were not met: + as_fn_error $? "Package requirements (lognorm >= 1.1.2) were not met: $LIBLOGNORM_PKG_ERRORS @@ -21855,7 +21811,6 @@ $as_echo "$as_me: Not running from git source" >&6;} fi - if test x$have_to_generate_man_pages = xyes; then ENABLE_GENERATE_MAN_PAGES_TRUE= ENABLE_GENERATE_MAN_PAGES_FALSE='#' @@ -22750,7 +22705,7 @@ # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" -This file was extended by rsyslog $as_me 8.12.0, which was +This file was extended by rsyslog $as_me 8.14.0, which was generated by GNU Autoconf 2.69. Invocation command line was CONFIG_FILES = $CONFIG_FILES @@ -22816,7 +22771,7 @@ cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ -rsyslog config.status 8.12.0 +rsyslog config.status 8.14.0 configured by $0, generated by GNU Autoconf 2.69, with options \\"\$ac_cs_config\\" diff -Nru rsyslog-8.12.0/configure.ac rsyslog-8.14.0/configure.ac --- rsyslog-8.12.0/configure.ac 2015-08-10 12:25:41.000000000 +0200 +++ rsyslog-8.14.0/configure.ac 2015-11-03 09:51:57.000000000 +0100 @@ -2,7 +2,7 @@ # Process this file with autoconf to produce a configure script. AC_PREREQ(2.61) -AC_INIT([rsyslog],[8.12.0],[rsyslog@lists.adiscon.com]) +AC_INIT([rsyslog],[8.14.0],[rsyslog@lists.adiscon.com]) AM_INIT_AUTOMAKE([subdir-objects]) m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) @@ -763,18 +763,13 @@ if test "x$enable_gnutls" = "xyes"; then PKG_CHECK_MODULES(GNUTLS, gnutls >= 1.4.0) AC_DEFINE([ENABLE_GNUTLS], [1], [Indicator that GnuTLS is present]) - AC_CHECK_LIB( - [gnutls], - [gnutls_global_init], - [ - AC_DEFINE(HAVE_LIB_GNUTLS, 1, [gnutls is available]) - ], - [AC_MSG_WARN([gnutls_global_init function missing or not detected])], - [] - ) + save_libs=$LIBS + LIBS="$LIBS $GNUTLS_LIBS" AC_CHECK_FUNCS(gnutls_certificate_set_retrieve_function,,) AC_CHECK_FUNCS(gnutls_certificate_type_set_priority,,) + LIBS=$save_libs fi + AM_CONDITIONAL(ENABLE_GNUTLS, test x$enable_gnutls = xyes) # libgcrypt support @@ -932,7 +927,7 @@ [enable_mmnormalize=no] ) if test "x$enable_mmnormalize" = "xyes"; then - PKG_CHECK_MODULES(LIBLOGNORM, lognorm >= 1.0.2) + PKG_CHECK_MODULES(LIBLOGNORM, lognorm >= 1.1.2) save_CFLAGS="$CFLAGS" save_LIBS="$LIBS" @@ -1655,7 +1650,6 @@ AC_MSG_NOTICE([Not running from git source]) fi - AM_CONDITIONAL(ENABLE_GENERATE_MAN_PAGES, test x$have_to_generate_man_pages = xyes) # rst2man diff -Nru rsyslog-8.12.0/contrib/imczmq/imczmq.c rsyslog-8.14.0/contrib/imczmq/imczmq.c --- rsyslog-8.12.0/contrib/imczmq/imczmq.c 2015-07-08 16:31:06.000000000 +0200 +++ rsyslog-8.14.0/contrib/imczmq/imczmq.c 2015-10-23 12:52:03.000000000 +0200 @@ -302,7 +302,7 @@ DBGPRINTF("imczmq: authtype is: %s\n", iconf->authType); /* if we are a CURVE server */ - if (!strcmp(iconf->authType, "CURVESERVER")) { + if ((iconf->authType != NULL) && (!strcmp(iconf->authType, "CURVESERVER"))) { iconf->is_server = true; @@ -329,7 +329,7 @@ } /* if we are a CURVE client */ - if (!strcmp(iconf->authType, "CURVECLIENT")) { + if ((iconf->authType != NULL) && (!strcmp(iconf->authType, "CURVECLIENT"))) { DBGPRINTF("imczmq: we are a curve client...\n"); iconf->is_server = false; @@ -360,21 +360,23 @@ /* if we have a ZMQ_SUB sock, subscribe to topics */ if (iconf->sockType == ZMQ_SUB) { - char topic[256]; - while (iconf->topicList) { - char *delimiter = strchr(iconf->topicList, ','); + iconf->is_server = false; + + char topic[256], *list = iconf->topicList; + while (list) { + char *delimiter = strchr(list, ','); if (!delimiter) { - delimiter = iconf->topicList + strlen (iconf->topicList); + delimiter = list + strlen (list); } - if (delimiter - iconf->topicList > 255) { + if (delimiter - list > 255) { errmsg.LogError(0, NO_ERRCODE, "iconf->topicList must be under 256 characters"); ABORT_FINALIZE(RS_RET_ERR); } - memcpy(topic, iconf->topicList, delimiter - iconf->topicList); - topic[delimiter - iconf->topicList] = 0; + memcpy(topic, list, delimiter - list); + topic[delimiter - list] = 0; zsock_set_subscribe(pData->sock, topic); @@ -382,12 +384,12 @@ break; } - iconf->topicList = delimiter + 1; + list = delimiter + 1; } } - /* FIXME: currently hard coded to bind */ - int rc = zsock_attach(pData->sock, (const char*)iconf->sockEndpoints, true); + int rc = zsock_attach(pData->sock, (const char*)iconf->sockEndpoints, + iconf->is_server); if (rc == -1) { errmsg.LogError(0, NO_ERRCODE, "zsock_attach to %s", iconf->sockEndpoints); diff -Nru rsyslog-8.12.0/contrib/omczmq/omczmq.c rsyslog-8.14.0/contrib/omczmq/omczmq.c --- rsyslog-8.12.0/contrib/omczmq/omczmq.c 2015-07-08 16:31:06.000000000 +0200 +++ rsyslog-8.14.0/contrib/omczmq/omczmq.c 2015-10-23 12:52:03.000000000 +0200 @@ -60,6 +60,7 @@ uchar *tplName; char *beacon; int beaconport; + char *topicList; } instanceData; typedef struct wrkrInstanceData { @@ -74,7 +75,8 @@ { "authtype", eCmdHdlrGetWord, 0 }, { "clientcertpath", eCmdHdlrGetWord, 0 }, { "servercertpath", eCmdHdlrGetWord, 0 }, - { "template", eCmdHdlrGetWord, 0 } + { "template", eCmdHdlrGetWord, 0 }, + { "topics", eCmdHdlrGetWord, 0 } }; static struct cnfparamblk actpblk = { @@ -136,7 +138,7 @@ } /* if we are a CURVE server */ - if (!strcmp(pData->authType, "CURVESERVER")) { + if ((pData->authType != NULL) && (!strcmp(pData->authType, "CURVESERVER"))) { DBGPRINTF("omczmq: we are a curve server...\n"); is_server = true; @@ -164,7 +166,7 @@ } /* if we are a CURVE client */ - if (!strcmp(pData->authType, "CURVECLIENT")) { + if ((pData->authType != NULL) && (!strcmp(pData->authType, "CURVECLIENT"))) { DBGPRINTF("omczmq: we are a curve client...\n"); is_server = false; @@ -193,6 +195,12 @@ zsock_set_curve_serverkey (pData->sock, server_key); } + /* if we are a PUB server */ + if (pData->sockType == ZMQ_PUB) { + DBGPRINTF("omczmq: we are a pub server...\n"); + is_server = true; + } + /* we default to CONNECT unless told otherwise */ int rc = zsock_attach(pData->sock, (const char*)pData->sockEndpoints, is_server); if (rc == -1) { @@ -213,14 +221,62 @@ CHKiRet(initCZMQ(pData)); } - /* send the log line */ - int rc = zstr_send(pData->sock, (char*)msg); + /* if we have a ZMQ_PUB sock and topics, send with topics */ + if (pData->sockType == ZMQ_PUB && pData->topicList) { + char topic[256], *list = pData->topicList; + int rc; + + while (list) { + char *delimiter = strchr(list, ','); + if (!delimiter) { + delimiter = list + strlen (list); + } - /* something is very wrong */ - if (rc == -1) { - errmsg.LogError(0, NO_ERRCODE, "omczmq: send of %s failed: %s", - msg, zmq_strerror(errno)); - ABORT_FINALIZE(RS_RET_ERR); + if (delimiter - list > 255) { + errmsg.LogError(0, NO_ERRCODE, + "pData->topicList must be under 256 characters"); + ABORT_FINALIZE(RS_RET_ERR); + } + + memcpy(topic, list, delimiter - list); + topic[delimiter - list] = 0; + + /* send the topic */ + rc = zstr_sendm(pData->sock, topic); + + /* something is very wrong */ + if (rc == -1) { + errmsg.LogError(0, NO_ERRCODE, "omczmq: send of topic %s failed" + ": %s", topic, zmq_strerror(errno)); + ABORT_FINALIZE(RS_RET_ERR); + } + + /* send the log line */ + rc = zstr_send(pData->sock, (char*)msg); + + /* something is very wrong */ + if (rc == -1) { + errmsg.LogError(0, NO_ERRCODE, "omczmq: send of %s failed: %s", + msg, zmq_strerror(errno)); + ABORT_FINALIZE(RS_RET_ERR); + } + + if (*delimiter == 0) { + break; + } + + list = delimiter + 1; + } + } else { + /* send the log line */ + int rc = zstr_send(pData->sock, (char*)msg); + + /* something is very wrong */ + if (rc == -1) { + errmsg.LogError(0, NO_ERRCODE, "omczmq: send of %s failed: %s", + msg, zmq_strerror(errno)); + ABORT_FINALIZE(RS_RET_ERR); + } } finalize_it: RETiRet; @@ -239,6 +295,7 @@ pData->authType = NULL; pData->clientCertPath = NULL; pData->serverCertPath = NULL; + pData->topicList = NULL; } @@ -278,6 +335,7 @@ free(pData->clientCertPath); free(pData->serverCertPath); free(pData->tplName); + free(pData->topicList); ENDfreeInstance @@ -388,6 +446,17 @@ pData->beaconport = atoi(es_str2cstr(pvals[i].val.d.estr, NULL)); } + /* get the subscription topics */ + else if(!strcmp(actpblk.descr[i].name, "topics")) { + if (pData->sockType != ZMQ_PUB) { + errmsg.LogError(0, RS_RET_CONFIG_ERROR, + "topics is invalid unless socktype is PUB"); + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } + + pData->topicList = es_str2cstr(pvals[i].val.d.estr, NULL); + } + /* the config has a bad option */ else { errmsg.LogError(0, NO_ERRCODE, "omczmq: %s is not a valid option", diff -Nru rsyslog-8.12.0/contrib/omhiredis/omhiredis.c rsyslog-8.14.0/contrib/omhiredis/omhiredis.c --- rsyslog-8.12.0/contrib/omhiredis/omhiredis.c 2015-07-08 16:31:06.000000000 +0200 +++ rsyslog-8.14.0/contrib/omhiredis/omhiredis.c 2015-11-03 08:14:26.000000000 +0100 @@ -1,5 +1,6 @@ /* omhiredis.c - * Copyright 2012 Talksum, Inc +* Copyright 2012 Talksum, Inc +* Copyright 2015 DigitalOcean, Inc * * This program is free software: you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public License @@ -16,7 +17,7 @@ * . * * Author: Brian Knox -* +* */ @@ -48,27 +49,37 @@ DEF_OMOD_STATIC_DATA DEFobjCurrIf(errmsg) +#define OMHIREDIS_MODE_TEMPLATE 0 +#define OMHIREDIS_MODE_QUEUE 1 +#define OMHIREDIS_MODE_PUBLISH 2 + /* our instance data. * this will be accessable * via pData */ typedef struct _instanceData { - uchar *server; /* redis server address */ - int port; /* redis port */ - uchar *tplName; /* template name */ + uchar *server; /* redis server address */ + int port; /* redis port */ + uchar *tplName; /* template name */ + char *modeDescription; /* mode description */ + int mode; /* mode constant */ + char *key; /* key for QUEUE and PUBLISH modes */ } instanceData; typedef struct wrkrInstanceData { - instanceData *pData; - redisContext *conn; /* redis connection */ + instanceData *pData; /* instanc data */ + redisContext *conn; /* redis connection */ redisReply **replies; /* array to hold replies from redis */ - int count; /* count of command sent for current batch */ + int count; /* count of command sent for current batch */ } wrkrInstanceData_t; static struct cnfparamdescr actpdescr[] = { { "server", eCmdHdlrGetWord, 0 }, { "serverport", eCmdHdlrInt, 0 }, - { "template", eCmdHdlrGetWord, 1 } + { "template", eCmdHdlrGetWord, 0 }, + { "mode", eCmdHdlrGetWord, 0 }, + { "key", eCmdHdlrGetWord, 0 }, }; + static struct cnfparamblk actpblk = { CNFPARAMBLK_VERSION, sizeof(actpdescr)/sizeof(struct cnfparamdescr), @@ -157,7 +168,21 @@ * increase our current pipeline count * by 1 and continue. */ int rc; - rc = redisAppendCommand(pWrkrData->conn, (char*)message); + switch(pWrkrData->pData->mode) { + case OMHIREDIS_MODE_TEMPLATE: + rc = redisAppendCommand(pWrkrData->conn, (char*)message); + break; + case OMHIREDIS_MODE_QUEUE: + rc = redisAppendCommand(pWrkrData->conn, "LPUSH %s %s", pWrkrData->pData->key, (char*)message); + break; + case OMHIREDIS_MODE_PUBLISH: + rc = redisAppendCommand(pWrkrData->conn, "PUBLISH %s %s", pWrkrData->pData->key, (char*)message); + break; + default: + dbgprintf("omhiredis: mode %d is invalid something is really wrong\n", pWrkrData->pData->mode); + ABORT_FINALIZE(RS_RET_ERR); + } + if (rc == REDIS_ERR) { errmsg.LogError(0, NO_ERRCODE, "omhiredis: %s", pWrkrData->conn->errstr); dbgprintf("omhiredis: %s\n", pWrkrData->conn->errstr); @@ -227,6 +252,9 @@ pData->server = NULL; pData->port = 6379; pData->tplName = NULL; + pData->mode = OMHIREDIS_MODE_TEMPLATE; + pData->modeDescription = "template"; + pData->key = NULL; } /* here is where the work to set up a new instance @@ -254,19 +282,51 @@ pData->port = (int) pvals[i].val.d.n, NULL; } else if(!strcmp(actpblk.descr[i].name, "template")) { pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "mode")) { + pData->modeDescription = es_str2cstr(pvals[i].val.d.estr, NULL); + if (!strcmp(pData->modeDescription, "template")) { + pData->mode = OMHIREDIS_MODE_TEMPLATE; + } else if (!strcmp(pData->modeDescription, "queue")) { + pData->mode = OMHIREDIS_MODE_QUEUE; + } else if (!strcmp(pData->modeDescription, "publish")) { + pData->mode = OMHIREDIS_MODE_PUBLISH; + } else { + dbgprintf("omhiredis: unsupported mode %s\n", actpblk.descr[i].name); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + } else if(!strcmp(actpblk.descr[i].name, "key")) { + pData->key = es_str2cstr(pvals[i].val.d.estr, NULL); } else { dbgprintf("omhiredis: program error, non-handled " "param '%s'\n", actpblk.descr[i].name); } } - if(pData->tplName == NULL) { - dbgprintf("omhiredis: action requires a template name"); - ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); - } + dbgprintf("omhiredis: checking config sanity\n"); - /* template string 0 is just a regular string */ - OMSRsetEntry(*ppOMSR, 0,(uchar*)pData->tplName, OMSR_NO_RQD_TPL_OPTS); + /* check config sanity for selected mode */ + switch(pData->mode) { + case OMHIREDIS_MODE_QUEUE: + case OMHIREDIS_MODE_PUBLISH: + if (pData->key == NULL) { + dbgprintf("omhiredis: mode %s requires a key\n", pData->modeDescription); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + if (pData->tplName == NULL) { + dbgprintf("omhiredis: using default RSYSLOG_ForwardFormat template\n"); + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*)strdup("RSYSLOG_ForwardFormat"), OMSR_NO_RQD_TPL_OPTS)); + } else { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*)pData->tplName, OMSR_NO_RQD_TPL_OPTS)); + } + break; + case OMHIREDIS_MODE_TEMPLATE: + if (pData->tplName == NULL) { + dbgprintf("omhiredis: selected mode requires template\n"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*)pData->tplName, OMSR_NO_RQD_TPL_OPTS)); + break; + } CODE_STD_FINALIZERnewActInst cnfparamvalsDestruct(pvals, &actpblk); diff -Nru rsyslog-8.12.0/contrib/omhiredis/README rsyslog-8.14.0/contrib/omhiredis/README --- rsyslog-8.12.0/contrib/omhiredis/README 2015-07-08 16:31:06.000000000 +0200 +++ rsyslog-8.14.0/contrib/omhiredis/README 2015-10-23 12:52:03.000000000 +0200 @@ -1,22 +1,70 @@ Redis Outplug Plugin using hiredis library -tested in Centos 6.2 and Archlinux +REQUIREMENTS: -BUILDING THIS PLUGIN -Requires the hiredis C client library: https://github.com/redis/hiredis/ +* hiredis ( https://github.com/redis/hiredis.git ) -in your /etc/rsyslog.conf, together with other modules: +USAGE: -Brian Knox +This plugin has three current "modes" that it supports: ---------------------------------------------------------------------------------------------- +1. "template" + +This is the original mode that the plugin supported. You use an rsyslog template +to construct a command that is sent directly to redis. This mode currently has +an issue dealing with strings that contain spaces. It's useful for doing things +like incrementing counters for statistics. + +``` +module(load="omhiredis") + +template( + name="simple_count" + type="string" + string="HINCRBY testcount %programname% 1") + +*.* action( + name="count_redis" + type="omhiredis" + mode="template" + template="simple_count" + ) +``` + +2. "queue" +The queue mode will LPUSH your message to a redis list. Unlike the template +mode, it handles full rsyslog messages properly. If a template is not +supplied, it will default to the RSYSLOG_ForwardFormat template. The "key" +parameter is required. + +``` +module(load="omhiredis") + +*.* action( + name="push_redis" + type="omhiredis" + mode="queue" + key="testqueue" + ) +``` + +3. "publish" +The publish mode will PUBLISH to a redis channel. Unlike the template mode, +it handles full rsyslog messages properly. If a template is not supplied, +it will default to the RSYSLOG_ForwardFormat template. The "key" +parameter is required and will be used for the publish channel. + +``` module(load="omhiredis") -template(name="simple_count" type="string" string="HINCRBY progcount %programname% 1") +*.* action( + name="publish_redis" + type="omhiredis" + mode="publish" + key="testpublish" + ) +``` -action(name="simple_count_redis" type="omhiredis" queue.type="FixedArray" queue.size="10000" queue.dequeuebatchsize="100" template="simple_count") ---------------------------------------------------------------------------------------------- -Note: dequeuebatchsize now sets the pipeline size for hiredis, allowing pipelining commands. -Note: this plugin will NOT handle full rsyslog messages properly yet. spaces in a property will - cause the redis command to be constructed improperly. a fix for this is in the works! +NOTES +* dequeuebatchsize now sets the pipeline size for hiredis, allowing pipelining commands. diff -Nru rsyslog-8.12.0/debian/changelog rsyslog-8.14.0/debian/changelog --- rsyslog-8.12.0/debian/changelog 2015-10-29 21:43:03.000000000 +0100 +++ rsyslog-8.14.0/debian/changelog 2015-12-02 13:53:47.000000000 +0100 @@ -1,3 +1,94 @@ +rsyslog (8.14.0-2ubuntu1) xenial; urgency=low + + * Merge from Debian unstable (LP: #1521673). Remaining changes: + - Run as rsyslog:rsyslog, set $FileOwner to syslog + - debian/rsyslog.conf: enable $RepeatedMsgReduction + to avoid bloating the syslog file. + - debian/50-default.conf: set of default rules for syslog (forwarded to + Debian #603160). remove file in postrm on purge. manage with ucf. + - Add disabled by default AppArmor profile: + + add debian/usr.sbin.rsyslogd profile + + debian/rules: use dh_apparmor to install profile before rsyslog is + restarted + + debian/control: suggests apparmor (>= 2.3) + + debian/rsyslog.install: install profile to /etc/apparmor.d + + debian/rsyslog.dirs: install /etc/apparmor.d/force-complain, + and /etc/apparmor.d/disable + + debian/rsyslog.preinst: disable profile on clean install or upgrades + from earlier than when we shipped the profile + + debian/control: Build-Depends on dh-apparmor + - debian/rsyslog.postinst: fix ownership of /var/spool/rsyslog. + - Adjust rsyslog init script to detect upstart, making the upstart + patches upstreamable to Debian. + - Add versioned dependency on lsb-base for the use of init_is_upstart. + * Dropped changes: + - debian/patches/fix-testbench-buffer-overflow-ftbs.patch : superseded upstream. + * debian/rules: filter out -Wl,-Bsymbolic-functions only, instead of + overriding all LDFLAGS. + * Drop rsyslog-mongodb package, depends on libmongo-client which is not + in main. + * Drop mmnormalize module, which depends on liblognorm from universe. + * Drop kafka package, depends on librdkafka from universe. + * Drop rsyslog-czmq, depends on libczmq-dev from universe. + * Build with --disable-liblogging-stdlog since liblogging-stdlog-dev is + in Universe + * Build with --disable-silent-rules to get useful build logs. + + -- Louis Bouchard Wed, 02 Dec 2015 12:09:39 +0100 + +rsyslog (8.14.0-2) unstable; urgency=medium + + * Remove logging to /dev/xconsole from the default rsyslog configuration. + It is a rarely used feature and causes rsyslog to repeatedly retry if + there is noone reading from /dev/xconsole. Instead ship the configuration + as example files and add instructions to README.Debian how one can + re-enable support for xconsole. (Closes: #745492, #783687) + * Remove section about sysklogd from README.Debian, sysklogd is not really + relevant anymore nowadays. + * Ship an example file for logging to the console on tty8 which can be + dropped into /etc/rsyslog.d instead of having it as comment in + /etc/rsyslog.conf. + + -- Michael Biebl Thu, 12 Nov 2015 22:11:11 +0100 + +rsyslog (8.14.0-1) unstable; urgency=medium + + * New upstream release. + * Rebase patches. + + -- Michael Biebl Sat, 07 Nov 2015 20:23:05 +0100 + +rsyslog (8.13.0-1) unstable; urgency=medium + + * New upstream release. + * Bump Build-Depends on liblognorm-dev to (>= 1.1.2) as per configure.ac. + * Rebase patches. + * Skip tests which are currently broken by the liblognorm update. According + to upstream those failing tests point at a bug in liblognorm, not rsyslog + itself. (Closes: #800873) + + -- Michael Biebl Sun, 25 Oct 2015 22:14:41 +0100 + +rsyslog (8.12.0-3) unstable; urgency=medium + + [ Louis Bouchard ] + * Fix FTBFS on various architectures caused by a buffer overflow in tcpflood + when running the test suite. + + -- Michael Biebl Wed, 16 Sep 2015 18:09:49 +0200 + +rsyslog (8.12.0-2) unstable; urgency=medium + + [ Michael Biebl ] + * Drop versioned Depends on initscripts. This dependency was added for the + /run transition in wheezy and is no longer required. + + [ Luca Boccassi ] + * Enable CZMQ plugins. (Closes: #798223) + * Fix NULL pointer dereference in imczmq and omczmq. + + -- Michael Biebl Tue, 15 Sep 2015 19:43:12 +0200 + rsyslog (8.12.0-1ubuntu3) xenial; urgency=medium * debian/rsyslog.logcheck.ignore.server: Suppress warning about duplicate diff -Nru rsyslog-8.12.0/debian/control rsyslog-8.14.0/debian/control --- rsyslog-8.12.0/debian/control 2015-09-01 08:56:48.000000000 +0200 +++ rsyslog-8.14.0/debian/control 2015-12-02 14:23:17.000000000 +0100 @@ -36,7 +36,6 @@ Depends: ${shlibs:Depends}, ${misc:Depends}, lsb-base (>= 4.1+Debian3), - initscripts (>= 2.88dsf-13.3), adduser, ucf Recommends: logrotate @@ -155,3 +154,13 @@ These plugins allow rsyslog to send and receive syslog messages via the RELP protocol. RELP ensures reliable transport over the network even on connection loss or if a peer becomes unavailable. + +Package: rsyslog-czmq +Architecture: any +Priority: extra +Depends: ${shlibs:Depends}, + ${misc:Depends}, + rsyslog (= ${binary:Version}) +Description: ZeroMQ protocol support for rsyslog via CZMQ + These plugins allows rsyslog to send and receive ZeroMQ syslog + messages via the CZMQ library. diff -Nru rsyslog-8.12.0/debian/examples/rsyslog.d/console.conf rsyslog-8.14.0/debian/examples/rsyslog.d/console.conf --- rsyslog-8.12.0/debian/examples/rsyslog.d/console.conf 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/debian/examples/rsyslog.d/console.conf 2015-11-12 22:11:24.000000000 +0100 @@ -0,0 +1,7 @@ +# I like to have messages displayed on the console, but only on a virtual +# console I usually leave idle. +# +daemon,mail.*;\ + news.=crit;news.=err;news.=notice;\ + *.=debug;*.=info;\ + *.=notice;*.=warn /dev/tty8 diff -Nru rsyslog-8.12.0/debian/examples/rsyslog.d/xconsole.conf rsyslog-8.14.0/debian/examples/rsyslog.d/xconsole.conf --- rsyslog-8.12.0/debian/examples/rsyslog.d/xconsole.conf 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/debian/examples/rsyslog.d/xconsole.conf 2015-11-12 22:11:24.000000000 +0100 @@ -0,0 +1,12 @@ +# The named pipe /dev/xconsole is for the `xconsole' utility. To use it, +# you must invoke `xconsole' with the `-file' option: +# +# $ xconsole -file /dev/xconsole [...] +# +# NOTE: adjust the list below, or you'll go crazy if you have a reasonably +# busy site.. +# +daemon.*;mail.*;\ + news.err;\ + *.=debug;*.=info;\ + *.=notice;*.=warn |/dev/xconsole diff -Nru rsyslog-8.12.0/debian/examples/tmpfiles.d/xconsole.conf rsyslog-8.14.0/debian/examples/tmpfiles.d/xconsole.conf --- rsyslog-8.12.0/debian/examples/tmpfiles.d/xconsole.conf 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/debian/examples/tmpfiles.d/xconsole.conf 2015-11-12 22:11:24.000000000 +0100 @@ -0,0 +1 @@ +p /dev/xconsole 0640 root adm diff -Nru rsyslog-8.12.0/debian/patches/Don-t-explicitly-link-tcpflood-against-lgcrypt.patch rsyslog-8.14.0/debian/patches/Don-t-explicitly-link-tcpflood-against-lgcrypt.patch --- rsyslog-8.12.0/debian/patches/Don-t-explicitly-link-tcpflood-against-lgcrypt.patch 2015-08-16 18:46:13.000000000 +0200 +++ rsyslog-8.14.0/debian/patches/Don-t-explicitly-link-tcpflood-against-lgcrypt.patch 2015-11-13 05:10:31.000000000 +0100 @@ -9,10 +9,10 @@ 1 file changed, 3 deletions(-) diff --git a/tests/Makefile.am b/tests/Makefile.am -index ce392da..fa00042 100644 +index 8f7ab58..299f797 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am -@@ -867,9 +867,6 @@ uxsockrcvr_LDADD = $(SOL_LIBS) +@@ -889,9 +889,6 @@ uxsockrcvr_LDADD = $(SOL_LIBS) tcpflood_SOURCES = tcpflood.c tcpflood_CPPFLAGS = $(PTHREADS_CFLAGS) $(GNUTLS_CFLAGS) tcpflood_LDADD = $(SOL_LIBS) $(PTHREADS_LIBS) $(GNUTLS_LIBS) diff -Nru rsyslog-8.12.0/debian/patches/fix-testbench-buffer-overflow-ftbs.patch rsyslog-8.14.0/debian/patches/fix-testbench-buffer-overflow-ftbs.patch --- rsyslog-8.12.0/debian/patches/fix-testbench-buffer-overflow-ftbs.patch 2015-09-02 15:00:39.000000000 +0200 +++ rsyslog-8.14.0/debian/patches/fix-testbench-buffer-overflow-ftbs.patch 1970-01-01 01:00:00.000000000 +0100 @@ -1,25 +0,0 @@ -Description: Fix potential buffer overflow detection - On some architectures, calculation of edLen may lead to a - negative value when the value returned by rand() is less - than extraDataLen away from RAND_MAX. The negative value - passed to memset() triggers a glibc error. Forcing the - cast of rand() to be unsigned avoid the return of a negative - value. - -Author: Louis Bouchard -Bug-Ubuntu: https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1464201 -Forwarded: https://github.com/rsyslog/rsyslog/issues/506 ---- -Index: rsyslog-8.12.0/tests/tcpflood.c -=================================================================== ---- rsyslog-8.12.0.orig/tests/tcpflood.c 2015-07-08 16:31:07.000000000 +0200 -+++ rsyslog-8.12.0/tests/tcpflood.c 2015-09-02 14:36:56.168433982 +0200 -@@ -398,7 +398,7 @@ - } - } else { - if(bRandomizeExtraData) -- edLen = ((long) rand() + extraDataLen) % extraDataLen + 1; -+ edLen = ((unsigned long) rand() + extraDataLen) % extraDataLen + 1; - else - edLen = extraDataLen; - memset(extraData, 'X', edLen); diff -Nru rsyslog-8.12.0/debian/patches/series rsyslog-8.14.0/debian/patches/series --- rsyslog-8.12.0/debian/patches/series 2015-09-02 15:00:39.000000000 +0200 +++ rsyslog-8.14.0/debian/patches/series 2015-12-01 16:53:53.000000000 +0100 @@ -1,3 +1,3 @@ Don-t-create-a-database.patch Don-t-explicitly-link-tcpflood-against-lgcrypt.patch -fix-testbench-buffer-overflow-ftbs.patch +Skip-broken-mmnormalize-tests.patch diff -Nru rsyslog-8.12.0/debian/patches/Skip-broken-mmnormalize-tests.patch rsyslog-8.14.0/debian/patches/Skip-broken-mmnormalize-tests.patch --- rsyslog-8.12.0/debian/patches/Skip-broken-mmnormalize-tests.patch 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/debian/patches/Skip-broken-mmnormalize-tests.patch 2015-11-12 22:11:24.000000000 +0100 @@ -0,0 +1,42 @@ +From: Michael Biebl +Date: Sun, 25 Oct 2015 21:29:27 +0100 +Subject: Skip broken mmnormalize tests + +Skip tests which are broken by the liblognorm update. + +See https://github.com/rsyslog/rsyslog/issues/546 +--- + tests/mmnormalize_tokenized.sh | 4 ++++ + tests/mmnormalize_variable.sh | 4 ++++ + 2 files changed, 8 insertions(+) + +diff --git a/tests/mmnormalize_tokenized.sh b/tests/mmnormalize_tokenized.sh +index e3f7243..10f45f8 100755 +--- a/tests/mmnormalize_tokenized.sh ++++ b/tests/mmnormalize_tokenized.sh +@@ -3,6 +3,10 @@ + # This file is part of the rsyslog project, released under ASL 2.0 + echo =============================================================================== + echo \[mmnormalize_tokenized.sh\]: test for mmnormalize tokenized field_type ++ ++echo "Currently broken, see https://github.com/rsyslog/rsyslog/issues/546" ++exit 77 ++ + . $srcdir/diag.sh init + . $srcdir/diag.sh startup mmnormalize_tokenized.conf + . $srcdir/diag.sh tcpflood -m 1 -I $srcdir/testsuites/tokenized_input +diff --git a/tests/mmnormalize_variable.sh b/tests/mmnormalize_variable.sh +index b86edef..203a30e 100755 +--- a/tests/mmnormalize_variable.sh ++++ b/tests/mmnormalize_variable.sh +@@ -3,6 +3,10 @@ + # This file is part of the rsyslog project, released under ASL 2.0 + echo =============================================================================== + echo \[mmnormalize_variable.sh\]: basic test for mmnormalize module variable-support ++ ++echo "Currently broken, see https://github.com/rsyslog/rsyslog/issues/546" ++exit 77 ++ + . $srcdir/diag.sh init + . $srcdir/diag.sh startup mmnormalize_variable.conf + . $srcdir/diag.sh tcpflood -m 1 -I $srcdir/testsuites/date_time_msg diff -Nru rsyslog-8.12.0/debian/README.Debian rsyslog-8.14.0/debian/README.Debian --- rsyslog-8.12.0/debian/README.Debian 2015-08-16 18:46:13.000000000 +0200 +++ rsyslog-8.14.0/debian/README.Debian 2015-11-13 05:10:31.000000000 +0100 @@ -35,22 +35,14 @@ (i.e. only if .0 is newer than .1). -Forwarding messages to sysklogd -------------------------------- -If you are forwarding messages from a rsyslog client to a sysklogd -server, it can lead to doubled hostnames in the syslog message on the -server side. The reason is a limitation in sysklogd which does not parse -the hostname in the syslog header (as defined by RFC 3164). See Debian -bug #514051 for more details. +Logging to xconsole +------------------- +The default rsyslog configuration no longer forwards any log messages to +/dev/xconsole. It is simple to re-enable support for xconsole though. For that, +the rsyslog package ships example files in /usr/share/doc/rsyslog/examples. -A simple workaround in rsyslog is, to define a custom template, which -does not include the hostname: +To create the /dev/xconsole pipe, copy examples/tmpfiles.d/xconsole.conf to +/etc/tmpfiles.d/ and then run "systemd-tmpfiles --create xconsole.conf". - $template sysklogd,"<%PRI%>%TIMESTAMP% %syslogtag%%msg%" - *.* @remote-host;sysklogd - -This creates a template called "sysklogd" and assigns it to the forward -rule when logging to "remote-host". - - - -- Michael Biebl Sat, 07 Feb 2009 18:54:37 +0100 +And finally copy examples/rsyslog.d/xconsole.conf to /etc/rsyslog.d/ and +restart the rsyslog service. diff -Nru rsyslog-8.12.0/debian/rsyslog-czmq.install rsyslog-8.14.0/debian/rsyslog-czmq.install --- rsyslog-8.12.0/debian/rsyslog-czmq.install 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/debian/rsyslog-czmq.install 2015-11-12 22:11:24.000000000 +0100 @@ -0,0 +1,2 @@ +usr/lib/rsyslog/omczmq.so +usr/lib/rsyslog/imczmq.so diff -Nru rsyslog-8.12.0/debian/rsyslog-czmq.README.Debian rsyslog-8.14.0/debian/rsyslog-czmq.README.Debian --- rsyslog-8.12.0/debian/rsyslog-czmq.README.Debian 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/debian/rsyslog-czmq.README.Debian 2015-11-12 22:11:24.000000000 +0100 @@ -0,0 +1,64 @@ +How to use rsyslog and CZMQ +=========================== + +Starting with version 8.6.0, rsyslog comes with output and input modules named +"imczmq" and "omczmq", allowing one to publish log messages via the ZeroMQ +protocol. More information about the protocol can be found on the website +http://zeromq.org/ + +Example configurations, taken from rsyslog upstream, follow. + +------------------------------------------------------------------------------- +module(load="imczmq") + +input( + type="imczmq" + endpoints="tcp://*:24555" + topics="topic1,topic2,topic3" + socktype="PULL" + authtype="CURVESERVER" + clientcertpath="/etc/curve.d/" + servercertpath="/etc/curve.d/example_curve_server_cert" +) +------------------------------------------------------------------------------- + +Explanation of options: + +type: type of action (imczmq for this plugin) +endpoints: comma delimited list of zeromq endpoints (see zeromq documentation) +socktype: zeromq socket type (currently supports PULL and SUB) +authtype: CURVECLIENT or CURVESERVER +clientcertpath: + if CURVECLIENT, this client's cert + if CURVESERVER, "*" for all, or a directory of allowed public certs +servercertpath: + if CURVECLIENT, the servers public cert you wish to connect to + if CURVESERVER, this servers cert + +------------------------------------------------------------------------------- +module(load="omczmq") + +action( + name="curve_server_socket" + type="omczmq" + endpoints="tcp://some.server.com:24445" + socktype="PUSH" + authtype="CURVECLIENT" + clientcertpath="/etc/curve.d/example_curve_client_cert" + servercertpath="/etc/curve.d/example_curve_server_cert" +) +------------------------------------------------------------------------------- + +Explanation of options: + +name: name of this action +type: type of action (omczmq for this plugin) +endpoints: comma delimited list of zeromq endpoints (see zeromq documentation) +socktype: zeromq socket type (currently supports PUSH and PUB) +authtype: CURVECLIENT or CURVESERVER +clientcertpath: + if CURVECLIENT, this client's cert + if CURVESERVER, "*" for all, or a directory of allowed public certs +servercertpath: + if CURVECLIENT, the servers public cert you wish to connect to + if CURVESERVER, this servers cert diff -Nru rsyslog-8.12.0/debian/rsyslog.examples rsyslog-8.14.0/debian/rsyslog.examples --- rsyslog-8.12.0/debian/rsyslog.examples 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/debian/rsyslog.examples 2015-11-12 22:11:24.000000000 +0100 @@ -0,0 +1 @@ +debian/examples/* diff -Nru rsyslog-8.12.0/debian/rsyslog.install rsyslog-8.14.0/debian/rsyslog.install --- rsyslog-8.12.0/debian/rsyslog.install 2015-09-01 08:56:48.000000000 +0200 +++ rsyslog-8.14.0/debian/rsyslog.install 2015-12-02 11:57:43.000000000 +0100 @@ -1,6 +1,5 @@ debian/rsyslog.conf etc/ debian/00rsyslog.conf usr/lib/tmpfiles.d/ -debian/xconsole.conf usr/lib/tmpfiles.d/ debian/50-default.conf /usr/share/rsyslog usr/sbin/ usr/share/man/ diff -Nru rsyslog-8.12.0/debian/rules rsyslog-8.14.0/debian/rules --- rsyslog-8.12.0/debian/rules 2015-09-01 08:56:48.000000000 +0200 +++ rsyslog-8.14.0/debian/rules 2015-12-02 14:23:34.000000000 +0100 @@ -7,7 +7,7 @@ DPKG_EXPORT_BUILDFLAGS = 1 #Avoid buiding specific packages that rely on Universe deps -export DH_OPTIONS = -Nrsyslog-mongodb -Nrsyslog-kafka +export DH_OPTIONS = -Nrsyslog-mongodb -Nrsyslog-kafka -Nrsyslog-czmq include /usr/share/dpkg/buildflags.mk diff -Nru rsyslog-8.12.0/debian/xconsole.conf rsyslog-8.14.0/debian/xconsole.conf --- rsyslog-8.12.0/debian/xconsole.conf 2015-08-16 18:46:13.000000000 +0200 +++ rsyslog-8.14.0/debian/xconsole.conf 1970-01-01 01:00:00.000000000 +0100 @@ -1 +0,0 @@ -p /dev/xconsole 0640 root adm diff -Nru rsyslog-8.12.0/grammar/rainerscript.c rsyslog-8.14.0/grammar/rainerscript.c --- rsyslog-8.12.0/grammar/rainerscript.c 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/grammar/rainerscript.c 2015-11-03 08:10:58.000000000 +0100 @@ -3747,7 +3747,7 @@ regex_t *re; DEFiRet; - if(func->nParams != 2) { + if(func->nParams < 2) { parser_errmsg("rsyslog logic error in line %d of file %s\n", __LINE__, __FILE__); FINALIZE; diff -Nru rsyslog-8.12.0/plugins/imfile/imfile.c rsyslog-8.14.0/plugins/imfile/imfile.c --- rsyslog-8.12.0/plugins/imfile/imfile.c 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/plugins/imfile/imfile.c 2015-11-03 08:30:31.000000000 +0100 @@ -368,8 +368,11 @@ } if(i < nWdmap) { /* we need to shift to make room for new entry */ + /* OLD dbgprintf("DDDD: imfile doing wdmap mmemmov(%d, %d, %d) for ADD\n", i,i+1,nWdmap-i); - memmove(wdmap + i, wdmap + i + 1, nWdmap - i); + memmove(wdmap + i, wdmap + i + 1, nWdmap - i); */ + dbgprintf("DDDD: imfile doing wdmap mmemmov(%d, %d, %d) for ADD\n", i + 1,i, sizeof(wd_map_t) * (nWdmap - i)); + memmove(wdmap + i + 1, wdmap + i, sizeof(wd_map_t) * (nWdmap - i)); } wdmap[i].wd = wd; wdmap[i].dirIdx = dirIdx; @@ -398,8 +401,11 @@ if(i < nWdmap-1) { /* we need to shift to delete it (see comment at wdmap definition) */ + /* OLD dbgprintf("DDDD: imfile doing wdmap mmemmov(%d, %d, %d) for DEL\n", i,i+1,nWdmap-i-1); - memmove(wdmap + i, wdmap + i+1, nWdmap - i-1); + memmove(wdmap + i, wdmap + i+1, nWdmap - i-1);*/ + dbgprintf("DDDD: imfile doing wdmap mmemmov(%d, %d, %d) for DEL\n", i, i + 1, sizeof(wd_map_t) * (nWdmap - i - 1)); + memmove(wdmap + i, wdmap + i + 1, sizeof(wd_map_t) * (nWdmap - i - 1)); } --nWdmap; dbgprintf("DDDD: imfile: wd %d deleted, was idx %d\n", wd, i); @@ -1539,8 +1545,14 @@ } } } else { + /* Duplicate static object as well, otherwise the configobject could be deleted later! */ + if(lstnDup(&pLstn, pLstn->pszBaseName) != RS_RET_OK) { + DBGPRINTF("imfile: in_setupFileWatchStatic failed to duplicate listener for '%s'\n", pLstn->pszFileName); + goto done; + } startLstnFile(pLstn); } +done: return; } /* setup our initial set of watches, based on user config */ @@ -1617,7 +1629,6 @@ uchar statefile[MAXFNAME]; uchar toDel[MAXFNAME]; int bDoRMState; - int ftIdx; uchar *statefn; DBGPRINTF("imfile: remove listener '%s', wd %d\n", pLstn->pszFileName, ev->wd); @@ -1630,15 +1641,9 @@ bDoRMState = 0; } pollFile(pLstn, NULL); /* one final try to gather data */ - /* do NOT delete listener data if the object is also linked to the - * configured table */ - ftIdx = fileTableSearchNoWildcard(&dirs[dirIdx].configured, pLstn->pszBaseName); - if(ftIdx == -1) { - DBGPRINTF("imfile: DELETING listener data for '%s' - '%s'\n", pLstn->pszBaseName, pLstn->pszFileName); - lstnDel(pLstn); - } else { - DBGPRINTF("imfile: NOT DELETING listener data for '%s' - '%s' - ftIdx = '%d' \n", pLstn->pszBaseName, pLstn->pszFileName, ftIdx); - } + /* delete listener data */ + DBGPRINTF("imfile: DELETING listener data for '%s' - '%s'\n", pLstn->pszBaseName, pLstn->pszFileName); + lstnDel(pLstn); fileTableDelFile(&dirs[dirIdx].active, pLstn); if(bDoRMState) { DBGPRINTF("imfile: unlinking '%s'\n", toDel); diff -Nru rsyslog-8.12.0/plugins/mmnormalize/mmnormalize.c rsyslog-8.14.0/plugins/mmnormalize/mmnormalize.c --- rsyslog-8.12.0/plugins/mmnormalize/mmnormalize.c 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/plugins/mmnormalize/mmnormalize.c 2015-10-23 12:52:03.000000000 +0200 @@ -6,7 +6,7 @@ * * File begun on 2010-01-01 by RGerhards * - * Copyright 2010-2013 Rainer Gerhards and Adiscon GmbH. + * Copyright 2010-2015 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -110,6 +110,13 @@ static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ +/* callback for liblognorm error messages */ +void +errCallBack(void __attribute__((unused)) *cookie, const char *msg, + size_t __attribute__((unused)) lenMsg) +{ + errmsg.LogError(0, RS_RET_ERR_LIBLOGNORM, "liblognorm error: %s", msg); +} /* to be called to build the liblognorm part of the instance ONCE ALL PARAMETERS ARE CORRECT * (and set within pData!). @@ -124,6 +131,7 @@ ABORT_FINALIZE(RS_RET_ERR_LIBLOGNORM_INIT); } ln_setCtxOpts(pData->ctxln, loadModConf->allow_regex); + ln_setErrMsgCB(pData->ctxln, errCallBack, NULL); if(ln_loadSamples(pData->ctxln, (char*) pData->rulebase) != 0) { errmsg.LogError(0, RS_RET_NO_RULEBASE, "error: normalization rulebase '%s' " "could not be loaded cannot activate action", pData->rulebase); diff -Nru rsyslog-8.12.0/plugins/omelasticsearch/omelasticsearch.c rsyslog-8.14.0/plugins/omelasticsearch/omelasticsearch.c --- rsyslog-8.12.0/plugins/omelasticsearch/omelasticsearch.c 2015-07-08 16:31:07.000000000 +0200 +++ rsyslog-8.14.0/plugins/omelasticsearch/omelasticsearch.c 2015-10-23 12:52:03.000000000 +0200 @@ -270,6 +270,8 @@ DBGPRINTF("omelasticsearch: checkConn() curl_easy_init() failed\n"); ABORT_FINALIZE(RS_RET_SUSPENDED); } + /* Fail on HTTP error */ + curl_easy_setopt(curl, CURLOPT_FAILONERROR, TRUE); /* Bodypart of request not needed, so set curl opt to nobody and httpget, otherwise lib-curl could sigsegv */ curl_easy_setopt(curl, CURLOPT_HTTPGET, TRUE); curl_easy_setopt(curl, CURLOPT_NOBODY, TRUE); @@ -1033,6 +1035,7 @@ case CURLE_COULDNT_RESOLVE_PROXY: case CURLE_COULDNT_CONNECT: case CURLE_WRITE_ERROR: + case CURLE_HTTP_RETURNED_ERROR: STATSCOUNTER_INC(indexHTTPReqFail, mutIndexHTTPReqFail); indexHTTPFail += nmsgs; DBGPRINTF("omelasticsearch: we are suspending ourselfs due " @@ -1137,6 +1140,8 @@ curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, curlResult); curl_easy_setopt(handle, CURLOPT_POST, 1); + curl_easy_setopt(handle, CURLOPT_FAILONERROR, TRUE); + pWrkrData->curlHandle = handle; pWrkrData->postHeader = header; diff -Nru rsyslog-8.12.0/plugins/omkafka/omkafka.c rsyslog-8.14.0/plugins/omkafka/omkafka.c --- rsyslog-8.12.0/plugins/omkafka/omkafka.c 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/plugins/omkafka/omkafka.c 2015-10-23 12:52:03.000000000 +0200 @@ -788,6 +788,7 @@ const int partition = getPartition(pData); rd_kafka_topic_t *rkt = NULL; pthread_rwlock_t *dynTopicLock = NULL; + int msg_enqueue_status = 0; DBGPRINTF("omkafka: trying to send: key:'%s', msg:'%s'\n", pData->key, msg); @@ -797,25 +798,32 @@ } else { rkt = pData->pTopic; } - if(rd_kafka_produce(rkt, partition, RD_KAFKA_MSG_F_COPY, - msg, strlen((char*)msg), pData->key, - pData->key == NULL ? 0 : strlen((char*)pData->key), - NULL) == -1) { + + msg_enqueue_status = rd_kafka_produce(rkt, partition, RD_KAFKA_MSG_F_COPY, + msg, strlen((char*)msg), pData->key, + pData->key == NULL ? 0 : strlen((char*)pData->key), + NULL); + if(msg_enqueue_status == -1) { errmsg.LogError(0, RS_RET_KAFKA_PRODUCE_ERR, "omkafka: Failed to produce to topic '%s' " "partition %d: %s\n", rd_kafka_topic_name(rkt), partition, rd_kafka_err2str(rd_kafka_errno2err(errno))); - STATSCOUNTER_INC(ctrKafkaFail, mutCtrKafkaFail); - ABORT_FINALIZE(RS_RET_KAFKA_PRODUCE_ERR); } const int callbacksCalled = rd_kafka_poll(pData->rk, 0); /* call callbacks */ if (pData->dynaTopic) { pthread_rwlock_unlock(dynTopicLock);/* dynamic topic can't be used beyond this pt */ } - DBGPRINTF("omkafka: kafka outqueue length: %d, callbacks called %d\n", - rd_kafka_outq_len(pData->rk), callbacksCalled); + rd_kafka_outq_len(pData->rk), callbacksCalled); + + if (msg_enqueue_status == -1) { + STATSCOUNTER_INC(ctrKafkaFail, mutCtrKafkaFail); + ABORT_FINALIZE(RS_RET_KAFKA_PRODUCE_ERR); + /* ABORT_FINALIZE isn't absolutely necessary as of now, + because this is the last line anyway, but its useful to ensure + correctness in case we add more stuff below this line at some point*/ + } finalize_it: DBGPRINTF("omkafka: writeKafka returned %d\n", iRet); diff -Nru rsyslog-8.12.0/plugins/omtesting/omtesting.c rsyslog-8.14.0/plugins/omtesting/omtesting.c --- rsyslog-8.12.0/plugins/omtesting/omtesting.c 2015-07-08 16:31:07.000000000 +0200 +++ rsyslog-8.14.0/plugins/omtesting/omtesting.c 2015-10-23 12:52:03.000000000 +0200 @@ -14,9 +14,9 @@ * * CURRENT SUPPORTED COMMANDS: * - * :omtesting:sleep + * :omtesting:sleep * - * Must be specified exactly as above. Keep in mind milliseconds are a millionth + * Must be specified exactly as above. Keep in mind microseconds are a millionth * of a second! * * NOTE: read comments in module-template.h to understand how this file @@ -69,7 +69,7 @@ mode; int bEchoStdout; int iWaitSeconds; - int iWaitUSeconds; /* milli-seconds (one million of a second, just to make sure...) */ + int iWaitUSeconds; /* micro-seconds (one millionth of a second, just to make sure...) */ int iCurrCallNbr; int iFailFrequency; int iResumeAfter; @@ -109,7 +109,7 @@ BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo - dbgprintf("Action delays rule by %d second(s) and %d millisecond(s)\n", + dbgprintf("Action delays rule by %d second(s) and %d microsecond(s)\n", pData->iWaitSeconds, pData->iWaitUSeconds); /* do nothing */ ENDdbgPrintInstInfo @@ -167,7 +167,7 @@ dbgprintf("sleep(%d, %d)\n", pData->iWaitSeconds, pData->iWaitUSeconds); tvSelectTimeout.tv_sec = pData->iWaitSeconds; - tvSelectTimeout.tv_usec = pData->iWaitUSeconds; /* milli seconds */ + tvSelectTimeout.tv_usec = pData->iWaitUSeconds; /* microseconds */ select(0, NULL, NULL, NULL, &tvSelectTimeout); RETiRet; } @@ -286,7 +286,7 @@ if(isspace(*p)) ++p; pData->iWaitSeconds = atoi((char*) szBuf); - /* parse milliseconds */ + /* parse microseconds */ for(i = 0 ; *p && !isspace(*p) && ((unsigned) i < sizeof(szBuf) - 1) ; ++i) { szBuf[i] = *p++; } diff -Nru rsyslog-8.12.0/runtime/lookup.c rsyslog-8.14.0/runtime/lookup.c --- rsyslog-8.12.0/runtime/lookup.c 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/runtime/lookup.c 2015-11-03 08:15:16.000000000 +0100 @@ -203,6 +203,7 @@ } free(pThis->d.strtab); pThis->d.strtab = newlu.d.strtab; /* hand table AND ALL STRINGS over! */ + pThis->nmemb = newlu.nmemb; pthread_rwlock_unlock(&pThis->rwlock); errmsg.LogError(0, RS_RET_OK, "lookup table '%s' reloaded from file '%s'", pThis->name, pThis->filename); diff -Nru rsyslog-8.12.0/runtime/Makefile.am rsyslog-8.14.0/runtime/Makefile.am --- rsyslog-8.12.0/runtime/Makefile.am 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/runtime/Makefile.am 2015-10-23 12:52:03.000000000 +0200 @@ -186,6 +186,7 @@ libgcry_la_SOURCES = libgcry.c libgcry_common.c libgcry.h libgcry_la_CPPFLAGS = $(RSRT_CFLAGS) $(LIBGCRYPT_CFLAGS) pkglib_LTLIBRARIES += lmcry_gcry.la + lmcry_gcry_la_DEPENDENCIES = librsyslog.la lmcry_gcry_la_SOURCES = lmcry_gcry.c lmcry_gcry.h lmcry_gcry_la_CPPFLAGS = $(RSRT_CFLAGS) $(LIBGCRYPT_CFLAGS) lmcry_gcry_la_LDFLAGS = -module -avoid-version \ diff -Nru rsyslog-8.12.0/runtime/Makefile.in rsyslog-8.14.0/runtime/Makefile.in --- rsyslog-8.12.0/runtime/Makefile.in 2015-08-10 12:25:58.000000000 +0200 +++ rsyslog-8.14.0/runtime/Makefile.in 2015-11-03 09:52:06.000000000 +0100 @@ -216,8 +216,6 @@ librsyslog_la-hashtable.lo librsyslog_la-hashtable_itr.lo \ ../librsyslog_la-outchannel.lo ../librsyslog_la-template.lo librsyslog_la_OBJECTS = $(am_librsyslog_la_OBJECTS) -@ENABLE_LIBGCRYPT_TRUE@lmcry_gcry_la_DEPENDENCIES = libgcry.la \ -@ENABLE_LIBGCRYPT_TRUE@ $(am__DEPENDENCIES_1) am__lmcry_gcry_la_SOURCES_DIST = lmcry_gcry.c lmcry_gcry.h @ENABLE_LIBGCRYPT_TRUE@am_lmcry_gcry_la_OBJECTS = \ @ENABLE_LIBGCRYPT_TRUE@ lmcry_gcry_la-lmcry_gcry.lo @@ -773,6 +771,7 @@ @ENABLE_GNUTLS_TRUE@lmnsd_gtls_la_LIBADD = $(GNUTLS_LIBS) @ENABLE_LIBGCRYPT_TRUE@libgcry_la_SOURCES = libgcry.c libgcry_common.c libgcry.h @ENABLE_LIBGCRYPT_TRUE@libgcry_la_CPPFLAGS = $(RSRT_CFLAGS) $(LIBGCRYPT_CFLAGS) +@ENABLE_LIBGCRYPT_TRUE@lmcry_gcry_la_DEPENDENCIES = librsyslog.la @ENABLE_LIBGCRYPT_TRUE@lmcry_gcry_la_SOURCES = lmcry_gcry.c lmcry_gcry.h @ENABLE_LIBGCRYPT_TRUE@lmcry_gcry_la_CPPFLAGS = $(RSRT_CFLAGS) $(LIBGCRYPT_CFLAGS) @ENABLE_LIBGCRYPT_TRUE@lmcry_gcry_la_LDFLAGS = -module -avoid-version \ diff -Nru rsyslog-8.12.0/runtime/msg.c rsyslog-8.14.0/runtime/msg.c --- rsyslog-8.12.0/runtime/msg.c 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/runtime/msg.c 2015-11-03 08:42:14.000000000 +0100 @@ -71,7 +71,7 @@ /* TODO: move the global variable root to the config object - had no time to to it * right now before vacation -- rgerhards, 2013-07-22 */ -static pthread_rwlock_t glblVars_rwlock; +static pthread_mutex_t glblVars_lock; struct json_object *global_var_root = NULL; /* static data */ @@ -405,6 +405,7 @@ static uchar * jsonPathGetLeaf(uchar *name, int lenName); static struct json_object *jsonDeepCopy(struct json_object *src); static json_bool jsonVarExtract(struct json_object* root, const char *key, struct json_object **value); +void getRawMsgAfterPRI(msg_t * const pM, uchar **pBuf, int *piLen); /* the locking and unlocking implementations: */ @@ -549,6 +550,8 @@ *pPropID = PROP_SYSLOGTAG; } else if(!strcmp((char*) pName, "rawmsg")) { *pPropID = PROP_RAWMSG; + } else if(!strcmp((char*) pName, "rawmsg-after-pri")) { + *pPropID = PROP_RAWMSG_AFTER_PRI; } else if(!strcmp((char*) pName, "inputname")) { *pPropID = PROP_INPUTNAME; } else if(!strcmp((char*) pName, "fromhost")) { @@ -1566,6 +1569,38 @@ } } +void +getRawMsgAfterPRI(msg_t * const pM, uchar **pBuf, int *piLen) +{ + if(pM == NULL) { + *pBuf= UCHAR_CONSTANT(""); + *piLen = 0; + } else { + if(pM->pszRawMsg == NULL) { + *pBuf= UCHAR_CONSTANT(""); + *piLen = 0; + } else { + /* unfortunately, pM->offAfterPRI seems NOT to be + * correct/consistent in all cases. imuxsock and imudp + * seem to have other values than imptcp. Testbench + * covers some of that. As a work-around, we caluculate + * the value ourselfes here. -- rgerhards, 2015-10-09 + */ + size_t offAfterPRI = 0; + if(pM->pszRawMsg[0] == '<') { /* do we have a PRI? */ + if(pM->pszRawMsg[2] == '>') + offAfterPRI = 3; + else if(pM->pszRawMsg[3] == '>') + offAfterPRI = 4; + else if(pM->pszRawMsg[4] == '>') + offAfterPRI = 5; + } + *pBuf = pM->pszRawMsg + offAfterPRI; + *piLen = pM->iLenRawMsg - offAfterPRI; + } + } +} + /* note: setMSGLen() is only for friends who really know what they * do. Setting an invalid length can be desasterous! @@ -2781,7 +2816,7 @@ } else if(pProp->id == PROP_LOCAL_VAR) { jroot = pMsg->localvars; } else if(pProp->id == PROP_GLOBAL_VAR) { - pthread_rwlock_rdlock(&glblVars_rwlock); + pthread_mutex_lock(&glblVars_lock); jroot = global_var_root; } else { DBGPRINTF("msgGetJSONPropVal; invalid property id %d\n", @@ -2806,7 +2841,7 @@ finalize_it: if(pProp->id == PROP_GLOBAL_VAR) - pthread_rwlock_unlock(&glblVars_rwlock); + pthread_mutex_unlock(&glblVars_lock); if(*pRes == NULL) { /* could not find any value, so set it to empty */ *pRes = (unsigned char*)""; @@ -2832,7 +2867,7 @@ } else if(pProp->id == PROP_LOCAL_VAR) { jroot = pMsg->localvars; } else if(pProp->id == PROP_GLOBAL_VAR) { - pthread_rwlock_rdlock(&glblVars_rwlock); + pthread_mutex_lock(&glblVars_lock); jroot = global_var_root; } else { DBGPRINTF("msgGetJSONPropJSON; invalid property id %d\n", @@ -2859,7 +2894,7 @@ if(pProp->id == PROP_GLOBAL_VAR) { if (*pjson != NULL) *pjson = jsonDeepCopy(*pjson); - pthread_rwlock_unlock(&glblVars_rwlock); + pthread_mutex_unlock(&glblVars_lock); } else { if (*pjson != NULL) json_object_get(*pjson); @@ -3134,6 +3169,9 @@ case PROP_RAWMSG: getRawMsg(pMsg, &pRes, &bufLen); break; + case PROP_RAWMSG_AFTER_PRI: + getRawMsgAfterPRI(pMsg, &pRes, &bufLen); + break; case PROP_INPUTNAME: getInputName(pMsg, &pRes, &bufLen); break; @@ -3279,11 +3317,13 @@ bufLen = 2; *pbMustBeFreed = 0; } else { + MsgLock(pMsg); if(pProp->id == PROP_CEE_ALL_JSON) { pRes = (uchar*)strdup(RS_json_object_to_json_string_ext(pMsg->json, JSON_C_TO_STRING_SPACED)); } else if(pProp->id == PROP_CEE_ALL_JSON_PLAIN) { pRes = (uchar*)strdup(RS_json_object_to_json_string_ext(pMsg->json, JSON_C_TO_STRING_PLAIN)); } + MsgUnlock(pMsg); *pbMustBeFreed = 1; } break; @@ -3573,7 +3613,7 @@ if(iTo > 0) --iTo; } - if(iFrom == 0 && iTo >= bufLen) { + if(iFrom == 0 && iTo >= bufLen && pTpe->data.field.options.bFixedWidth == 0) { /* in this case, the requested string is a superset of what we already have, * so there is no need to do any processing. This is a frequent case for size-limited * fields like TAG in the default forwarding template (so it is a useful optimization @@ -3581,8 +3621,10 @@ */ ; /*DO NOTHING*/ } else { - if(iTo > bufLen) /* iTo is very large, if no to-position is set in the template! */ - iTo = bufLen; + if(iTo > bufLen) /* iTo is very large, if no to-position is set in the template! */ + if (pTpe->data.field.options.bFixedWidth == 0) + iTo = bufLen; + iLen = iTo - iFrom + 1; /* the +1 is for an actual char, NOT \0! */ pBufStart = pBuf = MALLOC((iLen + 1) * sizeof(uchar)); if(pBuf == NULL) { @@ -3602,9 +3644,13 @@ } /* OK, we are at the begin - now let's copy... */ bufLen = iLen; - while(*pSb && iLen) { - *pBuf++ = *pSb; - ++pSb; + while(iLen) { + if (*pSb) { + *pBuf++ = *pSb; + ++pSb; + } else { + *pBuf++ = ' '; + } --iLen; } *pBuf = '\0'; @@ -4259,7 +4305,7 @@ } else if(name[0] == '.') { pjroot = &pM->localvars; } else if (name[0] == '/') { /* globl var */ - pthread_rwlock_wrlock(&glblVars_rwlock); + pthread_mutex_lock(&glblVars_lock); pjroot = &global_var_root; if (sharedReference) { given = json; @@ -4321,7 +4367,7 @@ finalize_it: if(name[0] == '/') - pthread_rwlock_unlock(&glblVars_rwlock); + pthread_mutex_unlock(&glblVars_lock); MsgUnlock(pM); RETiRet; } @@ -4342,7 +4388,7 @@ } else if(name[0] == '.') { jroot = &pM->localvars; } else if (name[0] == '/') { /* globl var */ - pthread_rwlock_wrlock(&glblVars_rwlock); + pthread_mutex_lock(&glblVars_lock); jroot = &global_var_root; } else { DBGPRINTF("Passed name %s is unknown kind of variable (It is not CEE, Local or Global variable).", name); @@ -4383,7 +4429,7 @@ finalize_it: if(name[0] == '/') - pthread_rwlock_unlock(&glblVars_rwlock); + pthread_mutex_unlock(&glblVars_lock); MsgUnlock(pM); RETiRet; } @@ -4569,7 +4615,7 @@ * rgerhards, 2008-01-04 */ BEGINObjClassInit(msg, 1, OBJ_IS_CORE_MODULE) - pthread_rwlock_init(&glblVars_rwlock, NULL); + pthread_mutex_init(&glblVars_lock, NULL); /* request objects we use */ CHKiRet(objUse(datetime, CORE_COMPONENT)); diff -Nru rsyslog-8.12.0/runtime/nsdsel_gtls.c rsyslog-8.14.0/runtime/nsdsel_gtls.c --- rsyslog-8.12.0/runtime/nsdsel_gtls.c 2015-07-08 16:31:07.000000000 +0200 +++ rsyslog-8.14.0/runtime/nsdsel_gtls.c 2015-10-23 12:52:03.000000000 +0200 @@ -198,7 +198,7 @@ pThis, pThis->iBufferRcvReady); FINALIZE; } - if(pNsdGTLS->rtryCall != gtlsRtry_None) { + if(pNsdGTLS->rtryCall == gtlsRtry_handshake) { CHKiRet(doRetry(pNsdGTLS)); /* we used this up for our own internal processing, so the socket * is not ready from the upper layer point of view. @@ -206,6 +206,14 @@ *pbIsReady = 0; FINALIZE; } + else if(pNsdGTLS->rtryCall == gtlsRtry_recv) { + iRet = doRetry(pNsdGTLS); + if(iRet == RS_RET_OK) { + *pbIsReady = 0; + FINALIZE; + } + } + /* now we must ensure that we do not fall back to PTCP if we have * done a "dummy" select. In that case, we know when the predicate * is not matched here, we do not have data available for this diff -Nru rsyslog-8.12.0/runtime/queue.c rsyslog-8.14.0/runtime/queue.c --- rsyslog-8.12.0/runtime/queue.c 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/runtime/queue.c 2015-10-23 12:52:03.000000000 +0200 @@ -743,6 +743,7 @@ ISOBJ_TYPE_assert(pStrm, strm); ISOBJ_TYPE_assert(pThis, qqueue); CHKiRet(strm.SetDir(pStrm, pThis->pszSpoolDir, pThis->lenSpoolDir)); + CHKiRet(strm.SetbSync(pStrm, pThis->bSyncQueueFiles)); finalize_it: RETiRet; } diff -Nru rsyslog-8.12.0/runtime/rsyslog.h rsyslog-8.14.0/runtime/rsyslog.h --- rsyslog-8.12.0/runtime/rsyslog.h 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/runtime/rsyslog.h 2015-10-23 12:52:03.000000000 +0200 @@ -441,6 +441,7 @@ RS_RET_KAFKA_PRODUCE_ERR = -2424,/**< error during Kafka produce function */ RS_RET_CONF_PARAM_INVLD = -2425,/**< config parameter is invalid */ RS_RET_KSI_ERR = -2426,/**< error in KSI subsystem */ + RS_RET_ERR_LIBLOGNORM = -2427,/**< cannot obtain liblognorm ctx */ /* RainerScript error messages (range 1000.. 1999) */ RS_RET_SYSVAR_NOT_FOUND = 1001, /**< system variable could not be found (maybe misspelled) */ diff -Nru rsyslog-8.12.0/runtime/stream.c rsyslog-8.14.0/runtime/stream.c --- rsyslog-8.12.0/runtime/stream.c 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/runtime/stream.c 2015-10-23 12:52:03.000000000 +0200 @@ -920,7 +920,7 @@ } /* if we are set to sync, we must obtain a file handle to the directory for fsync() purposes */ - if(pThis->bSync && !pThis->bIsTTY) { + if(pThis->bSync && !pThis->bIsTTY && pThis->pszDir != NULL) { pThis->fdDir = open((char*)pThis->pszDir, O_RDONLY | O_CLOEXEC | O_NOCTTY); if(pThis->fdDir == -1) { char errStr[1024]; diff -Nru rsyslog-8.12.0/runtime/typedefs.h rsyslog-8.14.0/runtime/typedefs.h --- rsyslog-8.12.0/runtime/typedefs.h 2015-07-08 16:31:07.000000000 +0200 +++ rsyslog-8.14.0/runtime/typedefs.h 2015-11-03 08:15:36.000000000 +0100 @@ -194,6 +194,7 @@ #define PROP_MSGID 22 #define PROP_PARSESUCCESS 23 #define PROP_JSONMESG 24 +#define PROP_RAWMSG_AFTER_PRI 25 #define PROP_SYS_NOW 150 #define PROP_SYS_YEAR 151 #define PROP_SYS_MONTH 152 diff -Nru rsyslog-8.12.0/.tarball-version rsyslog-8.14.0/.tarball-version --- rsyslog-8.12.0/.tarball-version 2015-08-10 12:28:25.000000000 +0200 +++ rsyslog-8.14.0/.tarball-version 2015-11-03 09:52:25.000000000 +0100 @@ -1 +1 @@ -8.12.0 +8.14.0 diff -Nru rsyslog-8.12.0/template.c rsyslog-8.14.0/template.c --- rsyslog-8.12.0/template.c 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/template.c 2015-10-23 12:52:03.000000000 +0200 @@ -89,6 +89,7 @@ { "regex.match", eCmdHdlrInt, 0 }, { "regex.submatch", eCmdHdlrInt, 0 }, { "droplastlf", eCmdHdlrBinary, 0 }, + { "fixedwidth", eCmdHdlrBinary, 0 }, { "mandatory", eCmdHdlrBinary, 0 }, { "spifno1stsp", eCmdHdlrBinary, 0 } }; @@ -755,6 +756,8 @@ pTpe->data.field.options.bSecPathReplace = 1; } else if(!strcmp((char*)Buf, "pos-end-relative")) { pTpe->data.field.options.bFromPosEndRelative = 1; + } else if(!strcmp((char*)Buf, "fixed-width")) { + pTpe->data.field.options.bFixedWidth = 1; } else if(!strcmp((char*)Buf, "csv")) { if(hasFormat(pTpe)) { errmsg.LogError(0, NO_ERRCODE, "error: can only specify " @@ -1413,6 +1416,7 @@ int topos = -1; int fieldnum = -1; int fielddelim = 9; /* default is HT (USACSII 9) */ + int fixedwidth = 0; int re_matchToUse = 0; int re_submatchToUse = 0; int bComplexProcessing = 0; @@ -1443,6 +1447,9 @@ } else if(!strcmp(pblkProperty.descr[i].name, "droplastlf")) { droplastlf = pvals[i].val.d.n; bComplexProcessing = 1; + } else if(!strcmp(pblkProperty.descr[i].name, "fixedwidth")) { + fixedwidth = pvals[i].val.d.n; + bComplexProcessing = 1; } else if(!strcmp(pblkProperty.descr[i].name, "mandatory")) { mandatory = pvals[i].val.d.n; } else if(!strcmp(pblkProperty.descr[i].name, "spifno1stsp")) { @@ -1657,6 +1664,7 @@ pTpe->data.field.options.bDropLastLF = droplastlf; pTpe->data.field.options.bSPIffNo1stSP = spifno1stsp; pTpe->data.field.options.bMandatory = mandatory; + pTpe->data.field.options.bFixedWidth = fixedwidth; pTpe->data.field.eCaseConv = caseconv; switch(formatType) { case F_NONE: diff -Nru rsyslog-8.12.0/template.h rsyslog-8.14.0/template.h --- rsyslog-8.12.0/template.h 2015-07-08 16:31:07.000000000 +0200 +++ rsyslog-8.14.0/template.h 2015-10-23 12:52:03.000000000 +0200 @@ -126,6 +126,7 @@ unsigned bJSONfr: 1; /* format field JSON *field* non escaped (n/v pair) */ unsigned bMandatory: 1; /* mandatory field - emit even if empty */ unsigned bFromPosEndRelative: 1;/* is From/To-Pos relative to end of string? */ + unsigned bFixedWidth: 1; /* space pad to toChar if string is shorter */ } options; /* options as bit fields */ } field; } data; diff -Nru rsyslog-8.12.0/tests/abort-uncleancfg-goodcfg.sh rsyslog-8.14.0/tests/abort-uncleancfg-goodcfg.sh --- rsyslog-8.12.0/tests/abort-uncleancfg-goodcfg.sh 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/tests/abort-uncleancfg-goodcfg.sh 2015-10-23 12:52:03.000000000 +0200 @@ -13,6 +13,6 @@ if [ ! -e rsyslog.out.log ] then echo "error: expected file does not exist" - . ./diag.sh error-exit 1 + . $srcdir/diag.sh error-exit 1 fi . $srcdir/diag.sh exit diff -Nru rsyslog-8.12.0/tests/diag.sh rsyslog-8.14.0/tests/diag.sh --- rsyslog-8.12.0/tests/diag.sh 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/tests/diag.sh 2015-11-03 08:15:16.000000000 +0100 @@ -36,11 +36,9 @@ #set -o xtrace #export RSYSLOG_DEBUG="debug nologfuncflow noprintmutexaction nostdout" #export RSYSLOG_DEBUGLOG="log" -CURRENT_TEST= TB_TIMEOUT_STARTSTOP=3000 # timeout for start/stop rsyslogd in tenths (!) of a second 3000 => 5 min case $1 in 'init') $srcdir/killrsyslog.sh # kill rsyslogd if it runs for some reason - basename $0 > CURRENT_TEST # save test name for auto-debugging if [ -z $RS_SORTCMD ]; then RS_SORTCMD=sort fi @@ -85,7 +83,7 @@ rm -f rsyslog.out.*.log rsyslog.random.data work-presort rsyslog.pipe rm -f rsyslog.input rsyslog.conf.tlscert stat-file1 rsyslog.empty rm -f rsyslog.errorfile - rm -f CURRENT_TEST HOSTNAME imfile-state:.-rsyslog.input + rm -f HOSTNAME imfile-state:.-rsyslog.input unset TCPFLOOD_EXTRA_OPTS echo ------------------------------------------------------------------------------- ;; @@ -103,12 +101,12 @@ 'startup') # start rsyslogd with default params. $2 is the config file name to use # returns only after successful startup, $3 is the instance (blank or 2!) $valgrind ../tools/rsyslogd -C -n -irsyslog$3.pid -M../runtime/.libs:../.libs -f$srcdir/testsuites/$2 & - $srcdir/diag.sh wait-startup $3 || . ./diag.sh error-exit $? + . $srcdir/diag.sh wait-startup $3 ;; 'startup-vg') # start rsyslogd with default params under valgrind control. $2 is the config file name to use # returns only after successful startup, $3 is the instance (blank or 2!) valgrind --log-fd=1 --error-exitcode=10 --malloc-fill=ff --free-fill=fe --leak-check=full ../tools/rsyslogd -C -n -irsyslog$3.pid -M../runtime/.libs:../.libs -f$srcdir/testsuites/$2 & - $srcdir/diag.sh wait-startup $3 || . ./diag.sh error-exit $? + . $srcdir/diag.sh wait-startup $3 echo startup-vg still running ;; 'startup-vg-noleak') # same as startup-vg, except that --leak-check is set to "none". This @@ -117,7 +115,7 @@ # they are platform-dependent. In that case, we can't test for leak checks # (obviously), but we can check for access violations, what still is useful. valgrind --log-fd=1 --error-exitcode=10 --malloc-fill=ff --free-fill=fe --leak-check=no ../tools/rsyslogd -C -n -irsyslog$3.pid -M../runtime/.libs:../.libs -f$srcdir/testsuites/$2 & - $srcdir/diag.sh wait-startup $3 || . ./diag.sh error-exit $? + . $srcdir/diag.sh wait-startup $3 echo startup-vg still running ;; 'wait-startup') # wait for rsyslogd startup ($2 is the instance) @@ -128,7 +126,7 @@ if test $i -gt $TB_TIMEOUT_STARTSTOP then echo "ABORT! Timeout waiting on startup (pid file)" - . ./diag.sh error-exit 1 + . $srcdir/diag.sh error-exit 1 fi done i=0 @@ -138,7 +136,7 @@ if test $i -gt $TB_TIMEOUT_STARTSTOP then echo "ABORT! Timeout waiting on startup ('started' file)" - . ./diag.sh error-exit 1 + . $srcdir/diag.sh error-exit 1 fi done echo "rsyslogd$2 started with pid " `cat rsyslog$2.pid` @@ -159,9 +157,8 @@ done if [ -e core.* ] then - echo "ABORT! core file exists, starting interactive shell" - bash - . ./diag.sh error-exit 1 + echo "ABORT! core file exists" + . $srcdir/diag.sh error-exit 1 fi ;; 'wait-shutdown-vg') # actually, we wait for rsyslog.pid to be deleted. $2 is the @@ -171,9 +168,8 @@ echo rsyslogd run exited with $RSYSLOGD_EXIT if [ -e vgcore.* ] then - echo "ABORT! core file exists, starting interactive shell" - bash - . ./diag.sh error-exit 1 + echo "ABORT! core file exists" + . $srcdir/diag.sh error-exit 1 fi ;; 'check-exit-vg') # wait for main message queue to be empty. $2 is the instance. @@ -186,25 +182,29 @@ 'get-mainqueuesize') # show the current main queue size if [ "$2" == "2" ] then - echo getmainmsgqueuesize | ./diagtalker -p13501 || . ./diag.sh error-exit $? + echo getmainmsgqueuesize | ./diagtalker -p13501 || . $srcdir/diag.sh error-exit $? else - echo getmainmsgqueuesize | ./diagtalker || . ./diag.sh error-exit $? + echo getmainmsgqueuesize | ./diagtalker || . $srcdir/diag.sh error-exit $? fi ;; 'wait-queueempty') # wait for main message queue to be empty. $2 is the instance. if [ "$2" == "2" ] then - echo WaitMainQueueEmpty | ./diagtalker -p13501 || . ./diag.sh error-exit $? + echo WaitMainQueueEmpty | ./diagtalker -p13501 || . $srcdir/diag.sh error-exit $? else - echo WaitMainQueueEmpty | ./diagtalker || . ./diag.sh error-exit $? + echo WaitMainQueueEmpty | ./diagtalker || . $srcdir/diag.sh error-exit $? fi ;; + 'issue-HUP') # shut rsyslogd down when main queue is empty. $2 is the instance. + kill -HUP `cat rsyslog$2.pid` + ./msleep 1000 + ;; 'shutdown-when-empty') # shut rsyslogd down when main queue is empty. $2 is the instance. if [ "$2" == "2" ] then echo Shutting down instance 2 fi - $srcdir/diag.sh wait-queueempty $2 || exit $? + . $srcdir/diag.sh wait-queueempty $2 ./msleep 1000 # wait a bit (think about slow testbench machines!) kill `cat rsyslog$2.pid` # note: we do not wait for the actual termination! @@ -219,12 +219,12 @@ if [ "$?" -ne "0" ]; then echo "error during tcpflood! see rsyslog.out.log.save for what was written" cp rsyslog.out.log rsyslog.out.log.save - . ./diag.sh error-exit 1 + . $srcdir/diag.sh error-exit 1 fi ;; 'injectmsg') # inject messages via our inject interface (imdiag) echo injecting $3 messages - echo injectmsg $2 $3 $4 $5 | ./diagtalker || . ./diag.sh error-exit $? + echo injectmsg $2 $3 $4 $5 | ./diagtalker || . $srcdir/diag.sh error-exit $? # TODO: some return state checking? (does it really make sense here?) ;; 'check-mainq-spool') # check if mainqueue spool files exist, if not abort (we just check .qi). @@ -233,7 +233,7 @@ if test ! -f test-spool/mainq.qi; then echo "error: mainq.qi does not exist where expected to do so!" ls -l test-spool - . ./diag.sh error-exit 1 + . $srcdir/diag.sh error-exit 1 fi ;; 'seq-check') # do the usual sequence check to see if everything was properly received. $2 is the instance. @@ -245,7 +245,7 @@ ./chkseq -fwork -s$2 -e$3 $4 $5 $6 $7 if [ "$?" -ne "0" ]; then echo "sequence error detected" - . ./diag.sh error-exit 1 + . $srcdir/diag.sh error-exit 1 fi ;; 'seq-check2') # do the usual sequence check to see if everything was properly received. This is @@ -258,7 +258,7 @@ ./chkseq -fwork2 -s$2 -e$3 $4 $5 $6 $7 if [ "$?" -ne "0" ]; then echo "sequence error detected" - . ./diag.sh error-exit 1 + . $srcdir/diag.sh error-exit 1 fi rm -f work2 ;; @@ -266,27 +266,27 @@ cat rsyslog.out.log | grep -qF "$2" if [ "$?" -ne "0" ]; then echo content-check failed - . ./diag.sh error-exit 1 + . $srcdir/diag.sh error-exit 1 fi ;; 'custom-content-check') cat $3 | grep -qF "$2" if [ "$?" -ne "0" ]; then echo content-check failed to find "'$2'" inside "'$3'" - . ./diag.sh error-exit 1 + . $srcdir/diag.sh error-exit 1 fi ;; 'content-pattern-check') cat rsyslog.out.log | grep -q "$2" if [ "$?" -ne "0" ]; then echo content-check failed, not every line matched pattern "'$2'" - . ./diag.sh error-exit 1 + . $srcdir/diag.sh error-exit 1 fi ;; 'assert-content-missing') cat rsyslog.out.log | grep -qF "$2" if [ "$?" -eq "0" ]; then - . ./diag.sh error-exit 1 + . $srcdir/diag.sh error-exit 1 fi ;; 'gzip-seq-check') # do the usual sequence check, but for gzip files @@ -298,14 +298,14 @@ ./chkseq -fwork -v -s$2 -e$3 $4 $5 $6 $7 if [ "$?" -ne "0" ]; then echo "sequence error detected" - . ./diag.sh error-exit 1 + . $srcdir/diag.sh error-exit 1 fi ;; 'nettester') # perform nettester-based tests # use -v for verbose output! ./nettester -t$2 -i$3 $4 if [ "$?" -ne "0" ]; then - . ./diag.sh error-exit 1 + . $srcdir/diag.sh error-exit 1 fi ;; 'setzcat') # find out name of zcat tool @@ -316,11 +316,11 @@ fi ;; 'generate-HOSTNAME') # generate the HOSTNAME file - . $srcdir/diag.sh startup gethostname.conf || . ./diag.sh error-exit $? - . $srcdir/diag.sh tcpflood -m1 -M "\"<128>\"" || . ./diag.sh error-exit $? + . $srcdir/diag.sh startup gethostname.conf + . $srcdir/diag.sh tcpflood -m1 -M "\"<128>\"" ./msleep 100 - . $srcdir/diag.sh shutdown-when-empty || . ./diag.sh error-exit $? # shut down rsyslogd when done processing messages - . $srcdir/diag.sh wait-shutdown || . ./diag.sh error-exit $? # we need to wait until rsyslogd is finished! + . $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages + . $srcdir/diag.sh wait-shutdown # we need to wait until rsyslogd is finished! ;; 'error-exit') # this is called if we had an error and need to abort. Here, we # try to gather as much information as possible. That's most important @@ -342,22 +342,19 @@ CORE= rm gdb.in fi - # check if we can re-run under valgrind and do so if possible - CURRENT_TEST=`cat CURRENT_TEST` - echo "" > CURRENT_TEST - if [ "x$CURRENT_TEST" != "x" ]; then - # OK, we have the testname and can re-run under valgrind - echo re-running under valgrind control - ./$CURRENT_TEST - # wait a little bit so that valgrind can finish - ./msleep 4000 - # next let's try us to get a debug log - RSYSLOG_DEBUG_SAVE=$RSYSLOG_DEBUG - export RSYSLOG_DEBUG="debug nologfuncflow noprintmutexaction" - ./$CURRENT_TEST - ./msleep 4000 - RSYSLOG_DEBUG=$RSYSLOG_DEBUG_SAVE - fi + + # OK, we have the testname and can re-run under valgrind + echo re-running under valgrind control + current_test="./$(basename $0)" # this path is probably wrong -- theinric + $current_test + # wait a little bit so that valgrind can finish + ./msleep 4000 + # next let's try us to get a debug log + RSYSLOG_DEBUG_SAVE=$RSYSLOG_DEBUG + export RSYSLOG_DEBUG="debug nologfuncflow noprintmutexaction" + $current_test + ./msleep 4000 + RSYSLOG_DEBUG=$RSYSLOG_DEBUG_SAVE rm IN_AUTO_DEBUG fi exit $2 diff -Nru rsyslog-8.12.0/tests/imfile-endregex.sh rsyslog-8.14.0/tests/imfile-endregex.sh --- rsyslog-8.12.0/tests/imfile-endregex.sh 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/tests/imfile-endregex.sh 2015-11-03 08:18:36.000000000 +0100 @@ -3,7 +3,12 @@ # This test mimics the test imfile-readmode2.sh, but works via # endmsg.regex. It's kind of a base test for the regex functionality. echo ====================================================================== -echo [imfile-endregex.sh] +# Check if inotify header exist +if [ -n "$(find /usr/include -name 'inotify.h' -print -quit)" ]; then + echo [imfile-endregex.sh] +else + exit 77 # no inotify available, skip this test +fi . $srcdir/diag.sh init . $srcdir/diag.sh startup imfile-endregex.conf diff -Nru rsyslog-8.12.0/tests/imfile-readmode2.sh rsyslog-8.14.0/tests/imfile-readmode2.sh --- rsyslog-8.12.0/tests/imfile-readmode2.sh 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/tests/imfile-readmode2.sh 2015-11-03 08:18:36.000000000 +0100 @@ -1,7 +1,12 @@ #!/bin/bash # This is part of the rsyslog testbench, licensed under ASL 2.0 echo ====================================================================== -echo [imfile-readmode2.sh] +# Check if inotify header exist +if [ -n "$(find /usr/include -name 'inotify.h' -print -quit)" ]; then + echo [imfile-readmode2.sh] +else + exit 77 # no inotify available, skip this test +fi . $srcdir/diag.sh init . $srcdir/diag.sh startup imfile-readmode2.conf diff -Nru rsyslog-8.12.0/tests/imfile-readmode2-with-persists-data-during-stop.sh rsyslog-8.14.0/tests/imfile-readmode2-with-persists-data-during-stop.sh --- rsyslog-8.12.0/tests/imfile-readmode2-with-persists-data-during-stop.sh 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/tests/imfile-readmode2-with-persists-data-during-stop.sh 2015-11-03 08:18:36.000000000 +0100 @@ -1,7 +1,12 @@ #!/bin/bash # This is part of the rsyslog testbench, licensed under ASL 2.0 echo ====================================================================== -echo [imfile-readmode2-with-persists-data-during-stop.sh] +# Check if inotify header exist +if [ -n "$(find /usr/include -name 'inotify.h' -print -quit)" ]; then + echo [imfile-readmode2-with-persists-data-during-stop.sh] +else + exit 77 # no inotify available, skip this test +fi . $srcdir/diag.sh init . $srcdir/diag.sh startup imfile-readmode2-with-persists-data-during-stop.conf diff -Nru rsyslog-8.12.0/tests/imfile-readmode2-with-persists.sh rsyslog-8.14.0/tests/imfile-readmode2-with-persists.sh --- rsyslog-8.12.0/tests/imfile-readmode2-with-persists.sh 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/tests/imfile-readmode2-with-persists.sh 2015-11-03 08:18:36.000000000 +0100 @@ -1,7 +1,12 @@ #!/bin/bash # This is part of the rsyslog testbench, licensed under ASL 2.0 echo ====================================================================== -echo [imfile-readmode2-with-persists.sh] +# Check if inotify header exist +if [ -n "$(find /usr/include -name 'inotify.h' -print -quit)" ]; then + echo [imfile-readmode2-with-persists.sh] +else + exit 77 # no inotify available, skip this test +fi . $srcdir/diag.sh init . $srcdir/diag.sh startup imfile-readmode2-with-persists.conf diff -Nru rsyslog-8.12.0/tests/imtcp-tls-basic-vg.sh rsyslog-8.14.0/tests/imtcp-tls-basic-vg.sh --- rsyslog-8.12.0/tests/imtcp-tls-basic-vg.sh 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/tests/imtcp-tls-basic-vg.sh 2015-11-03 08:34:13.000000000 +0100 @@ -7,7 +7,7 @@ echo \$DefaultNetstreamDriverCAFile $srcdir/tls-certs/ca.pem >rsyslog.conf.tlscert echo \$DefaultNetstreamDriverCertFile $srcdir/tls-certs/cert.pem >>rsyslog.conf.tlscert echo \$DefaultNetstreamDriverKeyFile $srcdir/tls-certs/key.pem >>rsyslog.conf.tlscert -. $srcdir/diag.sh startup-vg imtcp-tls-basic.conf +. $srcdir/diag.sh startup-vg-noleak imtcp-tls-basic.conf . $srcdir/diag.sh tcpflood -p13514 -m50000 -Ttls -Z$srcdir/tls-certs/cert.pem -z$srcdir/tls-certs/key.pem . $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages . $srcdir/diag.sh wait-shutdown-vg diff -Nru rsyslog-8.12.0/tests/lookup_table.sh rsyslog-8.14.0/tests/lookup_table.sh --- rsyslog-8.12.0/tests/lookup_table.sh 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/tests/lookup_table.sh 2015-11-03 08:15:16.000000000 +0100 @@ -0,0 +1,24 @@ +#!/bin/bash +# added 2015-09-30 by singh.janmejay +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[lookup_table_reload.sh\]: test for lookup-table and HUP based reloading of it +. $srcdir/diag.sh init +cp $srcdir/testsuites/xlate.lkp_tbl $srcdir/xlate.lkp_tbl +. $srcdir/diag.sh startup lookup_table.conf +. $srcdir/diag.sh injectmsg 0 3 +. $srcdir/diag.sh wait-queueempty +. $srcdir/diag.sh content-check "msgnum:00000000: foo_old" +. $srcdir/diag.sh content-check "msgnum:00000001: bar_old" +. $srcdir/diag.sh assert-content-missing "baz" +cp $srcdir/testsuites/xlate_more.lkp_tbl $srcdir/xlate.lkp_tbl +. $srcdir/diag.sh issue-HUP +. $srcdir/diag.sh injectmsg 0 3 +echo doing shutdown +. $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +. $srcdir/diag.sh wait-shutdown +. $srcdir/diag.sh content-check "msgnum:00000000: foo_new" +. $srcdir/diag.sh content-check "msgnum:00000001: bar_new" +. $srcdir/diag.sh content-check "msgnum:00000002: baz" +. $srcdir/diag.sh exit diff -Nru rsyslog-8.12.0/tests/Makefile.am rsyslog-8.14.0/tests/Makefile.am --- rsyslog-8.12.0/tests/Makefile.am 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/tests/Makefile.am 2015-11-03 08:15:58.000000000 +0100 @@ -24,6 +24,7 @@ fac_invld2.sh \ fac_invld3.sh \ fac_invld4_rfc5424.sh \ + rawmsg-after-pri.sh \ rfc5424parser.sh \ tcp_forwarding_tpl.sh \ tcp_forwarding_dflt_tpl.sh \ @@ -117,6 +118,11 @@ rscript_ruleset_call.sh \ rscript_set_modify.sh \ rscript_unaffected_reset.sh \ + rscript_replace_complex.sh \ + rscript_wrap2.sh \ + rscript_wrap3.sh \ + rscript_re_extract.sh \ + rscript_re_match.sh \ rs_optimizer_pri.sh \ cee_simple.sh \ cee_diskqueue.sh \ @@ -125,6 +131,7 @@ incltest_dir_wildcard.sh \ incltest_dir_empty_wildcard.sh \ linkedlistqueue.sh \ + lookup_table.sh \ key_dereference_on_uninitialized_variable_space.sh @@ -177,10 +184,7 @@ imptcp_no_octet_counted.sh \ imptcp_spframingfix.sh \ rscript_random.sh \ - rscript_replace.sh \ - rscript_replace_complex.sh \ - rscript_wrap2.sh \ - rscript_wrap3.sh + rscript_replace.sh if HAVE_VALGRIND TESTS += \ imptcp_conndrop-vg.sh @@ -215,12 +219,14 @@ endif if ENABLE_MMNORMALIZE +if ENABLE_IMPTCP TESTS += \ mmnormalize_variable.sh \ mmnormalize_tokenized.sh \ mmnormalize_regex_defaulted.sh \ mmnormalize_regex_disabled.sh +endif if LOGNORM_REGEX_SUPPORTED TESTS += \ mmnormalize_regex.sh @@ -228,12 +234,15 @@ endif if ENABLE_MMJSONPARSE +if ENABLE_IMPTCP TESTS += \ mmjsonparse_simple.sh \ mmjsonparse_cim.sh \ json_array_subscripting.sh \ json_array_looping.sh \ - json_nonarray_looping.sh \ + json_nonarray_looping.sh +endif +TESTS += \ stop_when_array_has_element.sh endif @@ -482,6 +491,8 @@ testsuites/fac_invld3.conf \ fac_invld4_rfc5424.sh \ testsuites/fac_invld4_rfc5424.conf \ + rawmsg-after-pri.sh \ + testsuites/rawmsg-after-pri.conf \ rs_optimizer_pri.sh \ testsuites/rs_optimizer_pri.conf \ rscript_prifilt.sh \ @@ -802,7 +813,10 @@ mysql-actq-mt-withpause-vg.sh \ testsuites/mysql-actq-mt.conf \ mmpstrucdata.sh \ + mmpstrucdata-vg.sh \ testsuites/mmpstrucdata.conf \ + mmpstrucdata-invalid-vg.sh \ + testsuites/mmpstrucdata-invalid.conf \ libdbi-basic-vg.sh \ mmnormalize_variable.sh \ mmnormalize_tokenized.sh \ @@ -846,6 +860,14 @@ testsuites/stop_when_array_has_element.conf \ key_dereference_on_uninitialized_variable_space.sh \ testsuites/key_dereference_on_uninitialized_variable_space.conf \ + rscript_re_extract.sh \ + testsuites/rscript_re_extract.conf \ + rscript_re_match.sh \ + testsuites/rscript_re_match.conf \ + lookup_table.sh \ + testsuites/lookup_table.conf \ + testsuites/xlate.lkp_tbl \ + testsuites/xlate_more.lkp_tbl \ cfg.sh # TODO: re-enable diff -Nru rsyslog-8.12.0/tests/Makefile.in rsyslog-8.14.0/tests/Makefile.in --- rsyslog-8.12.0/tests/Makefile.in 2015-08-10 12:25:58.000000000 +0200 +++ rsyslog-8.14.0/tests/Makefile.in 2015-11-03 09:52:06.000000000 +0100 @@ -91,7 +91,8 @@ @ENABLE_TESTBENCH_TRUE@ fac_ftp.sh fac_ntp.sh fac_uucp.sh \ @ENABLE_TESTBENCH_TRUE@ fac_invld1.sh fac_invld2.sh \ @ENABLE_TESTBENCH_TRUE@ fac_invld3.sh fac_invld4_rfc5424.sh \ -@ENABLE_TESTBENCH_TRUE@ rfc5424parser.sh tcp_forwarding_tpl.sh \ +@ENABLE_TESTBENCH_TRUE@ rawmsg-after-pri.sh rfc5424parser.sh \ +@ENABLE_TESTBENCH_TRUE@ tcp_forwarding_tpl.sh \ @ENABLE_TESTBENCH_TRUE@ tcp_forwarding_dflt_tpl.sh \ @ENABLE_TESTBENCH_TRUE@ arrayqueue.sh global_vars.sh \ @ENABLE_TESTBENCH_TRUE@ da-mainmsg-q.sh validation-run.sh \ @@ -155,12 +156,15 @@ @ENABLE_TESTBENCH_TRUE@ rscript_ruleset_call.sh \ @ENABLE_TESTBENCH_TRUE@ rscript_set_modify.sh \ @ENABLE_TESTBENCH_TRUE@ rscript_unaffected_reset.sh \ -@ENABLE_TESTBENCH_TRUE@ rs_optimizer_pri.sh cee_simple.sh \ -@ENABLE_TESTBENCH_TRUE@ cee_diskqueue.sh incltest.sh \ -@ENABLE_TESTBENCH_TRUE@ incltest_dir.sh \ +@ENABLE_TESTBENCH_TRUE@ rscript_replace_complex.sh \ +@ENABLE_TESTBENCH_TRUE@ rscript_wrap2.sh rscript_wrap3.sh \ +@ENABLE_TESTBENCH_TRUE@ rscript_re_extract.sh \ +@ENABLE_TESTBENCH_TRUE@ rscript_re_match.sh rs_optimizer_pri.sh \ +@ENABLE_TESTBENCH_TRUE@ cee_simple.sh cee_diskqueue.sh \ +@ENABLE_TESTBENCH_TRUE@ incltest.sh incltest_dir.sh \ @ENABLE_TESTBENCH_TRUE@ incltest_dir_wildcard.sh \ @ENABLE_TESTBENCH_TRUE@ incltest_dir_empty_wildcard.sh \ -@ENABLE_TESTBENCH_TRUE@ linkedlistqueue.sh \ +@ENABLE_TESTBENCH_TRUE@ linkedlistqueue.sh lookup_table.sh \ @ENABLE_TESTBENCH_TRUE@ key_dereference_on_uninitialized_variable_space.sh \ @ENABLE_TESTBENCH_TRUE@ $(am__append_1) $(am__append_2) \ @ENABLE_TESTBENCH_TRUE@ $(am__append_3) $(am__append_4) \ @@ -173,7 +177,7 @@ @ENABLE_TESTBENCH_TRUE@ $(am__append_17) $(am__append_18) \ @ENABLE_TESTBENCH_TRUE@ $(am__append_19) $(am__append_20) \ @ENABLE_TESTBENCH_TRUE@ $(am__append_21) $(am__append_22) \ -@ENABLE_TESTBENCH_TRUE@ $(am__append_23) +@ENABLE_TESTBENCH_TRUE@ $(am__append_23) $(am__append_24) @ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@am__append_1 = \ @ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ discard-rptdmsg-vg.sh \ @ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ discard-allmark-vg.sh \ @@ -215,10 +219,7 @@ @ENABLE_IMPTCP_TRUE@@ENABLE_TESTBENCH_TRUE@ imptcp_no_octet_counted.sh \ @ENABLE_IMPTCP_TRUE@@ENABLE_TESTBENCH_TRUE@ imptcp_spframingfix.sh \ @ENABLE_IMPTCP_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_random.sh \ -@ENABLE_IMPTCP_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_replace.sh \ -@ENABLE_IMPTCP_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_replace_complex.sh \ -@ENABLE_IMPTCP_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_wrap2.sh \ -@ENABLE_IMPTCP_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_wrap3.sh +@ENABLE_IMPTCP_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_replace.sh @ENABLE_IMPTCP_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@am__append_7 = \ @ENABLE_IMPTCP_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ imptcp_conndrop-vg.sh @@ -244,21 +245,23 @@ @ENABLE_MMPSTRUCDATA_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ mmpstrucdata-vg.sh \ @ENABLE_MMPSTRUCDATA_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ mmpstrucdata-invalid-vg.sh -@ENABLE_MMNORMALIZE_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_12 = \ -@ENABLE_MMNORMALIZE_TRUE@@ENABLE_TESTBENCH_TRUE@ mmnormalize_variable.sh \ -@ENABLE_MMNORMALIZE_TRUE@@ENABLE_TESTBENCH_TRUE@ mmnormalize_tokenized.sh \ -@ENABLE_MMNORMALIZE_TRUE@@ENABLE_TESTBENCH_TRUE@ mmnormalize_regex_defaulted.sh \ -@ENABLE_MMNORMALIZE_TRUE@@ENABLE_TESTBENCH_TRUE@ mmnormalize_regex_disabled.sh +@ENABLE_IMPTCP_TRUE@@ENABLE_MMNORMALIZE_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_12 = \ +@ENABLE_IMPTCP_TRUE@@ENABLE_MMNORMALIZE_TRUE@@ENABLE_TESTBENCH_TRUE@ mmnormalize_variable.sh \ +@ENABLE_IMPTCP_TRUE@@ENABLE_MMNORMALIZE_TRUE@@ENABLE_TESTBENCH_TRUE@ mmnormalize_tokenized.sh \ +@ENABLE_IMPTCP_TRUE@@ENABLE_MMNORMALIZE_TRUE@@ENABLE_TESTBENCH_TRUE@ mmnormalize_regex_defaulted.sh \ +@ENABLE_IMPTCP_TRUE@@ENABLE_MMNORMALIZE_TRUE@@ENABLE_TESTBENCH_TRUE@ mmnormalize_regex_disabled.sh @ENABLE_MMNORMALIZE_TRUE@@ENABLE_TESTBENCH_TRUE@@LOGNORM_REGEX_SUPPORTED_TRUE@am__append_13 = \ @ENABLE_MMNORMALIZE_TRUE@@ENABLE_TESTBENCH_TRUE@@LOGNORM_REGEX_SUPPORTED_TRUE@ mmnormalize_regex.sh -@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_14 = \ -@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@ mmjsonparse_simple.sh \ -@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@ mmjsonparse_cim.sh \ -@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@ json_array_subscripting.sh \ -@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@ json_array_looping.sh \ -@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@ json_nonarray_looping.sh \ +@ENABLE_IMPTCP_TRUE@@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_14 = \ +@ENABLE_IMPTCP_TRUE@@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@ mmjsonparse_simple.sh \ +@ENABLE_IMPTCP_TRUE@@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@ mmjsonparse_cim.sh \ +@ENABLE_IMPTCP_TRUE@@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@ json_array_subscripting.sh \ +@ENABLE_IMPTCP_TRUE@@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@ json_array_looping.sh \ +@ENABLE_IMPTCP_TRUE@@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@ json_nonarray_looping.sh + +@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_15 = \ @ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@ stop_when_array_has_element.sh @@ -267,23 +270,23 @@ #sndrcv_tls_anon.sh \ #sndrcv_tls_anon_rebind.sh \ #imtcp-tls-basic.sh -@ENABLE_GNUTLS_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@am__append_15 = \ +@ENABLE_GNUTLS_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@am__append_16 = \ @ENABLE_GNUTLS_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ imtcp-tls-basic-vg.sh \ @ENABLE_GNUTLS_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ imtcp_conndrop_tls-vg.sh -@ENABLE_OMUXSOCK_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_16 = uxsock_simple.sh -@ENABLE_RELP_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_17 = sndrcv_relp.sh -@ENABLE_OMUDPSPOOF_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_18 = \ +@ENABLE_OMUXSOCK_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_17 = uxsock_simple.sh +@ENABLE_RELP_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_18 = sndrcv_relp.sh +@ENABLE_OMUDPSPOOF_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_19 = \ @ENABLE_OMUDPSPOOF_TRUE@@ENABLE_TESTBENCH_TRUE@ sndrcv_omudpspoof.sh \ @ENABLE_OMUDPSPOOF_TRUE@@ENABLE_TESTBENCH_TRUE@ sndrcv_omudpspoof_nonstdpt.sh -@ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_19 = \ +@ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_20 = \ @ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@ omod-if-array.sh \ @ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@ threadingmq.sh \ @ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@ threadingmqaq.sh \ @ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@ badqi.sh -@ENABLE_IMPTCP_TRUE@@ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_20 = \ +@ENABLE_IMPTCP_TRUE@@ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_21 = \ @ENABLE_IMPTCP_TRUE@@ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@ tabescape_dflt.sh \ @ENABLE_IMPTCP_TRUE@@ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@ tabescape_off.sh \ @ENABLE_IMPTCP_TRUE@@ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@ timestamp.sh \ @@ -292,7 +295,7 @@ @ENABLE_IMPTCP_TRUE@@ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@ parsertest.sh \ @ENABLE_IMPTCP_TRUE@@ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@ fieldtest.sh -@ENABLE_OMRULESET_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_21 = \ +@ENABLE_OMRULESET_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_22 = \ @ENABLE_OMRULESET_TRUE@@ENABLE_TESTBENCH_TRUE@ omruleset.sh \ @ENABLE_OMRULESET_TRUE@@ENABLE_TESTBENCH_TRUE@ omruleset-queue.sh @@ -300,19 +303,19 @@ # random.sh is temporarily disabled as it needs some work # to rsyslog core to complete in reasonable time #TESTS += random.sh -@ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_22 = \ +@ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_23 = \ @ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@ imfile-basic.sh \ @ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@ imfile-readmode2.sh \ @ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@ imfile-readmode2-with-persists-data-during-stop.sh \ @ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@ imfile-readmode2-with-persists.sh \ @ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@ imfile-endregex.sh -@ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@am__append_23 = \ +@ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@am__append_24 = \ @ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ imfile-basic-vg.sh \ @ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ imfile-endregex-vg.sh \ @ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ imfile-readmode2-vg.sh -@ENABLE_GNUTLS_TRUE@am__append_24 = -lgcrypt +@ENABLE_GNUTLS_TRUE@am__append_25 = -lgcrypt subdir = tests DIST_COMMON = $(srcdir)/Makefile.in $(srcdir)/Makefile.am \ $(top_srcdir)/depcomp $(top_srcdir)/test-driver README @@ -1002,6 +1005,8 @@ testsuites/fac_invld3.conf \ fac_invld4_rfc5424.sh \ testsuites/fac_invld4_rfc5424.conf \ + rawmsg-after-pri.sh \ + testsuites/rawmsg-after-pri.conf \ rs_optimizer_pri.sh \ testsuites/rs_optimizer_pri.conf \ rscript_prifilt.sh \ @@ -1322,7 +1327,10 @@ mysql-actq-mt-withpause-vg.sh \ testsuites/mysql-actq-mt.conf \ mmpstrucdata.sh \ + mmpstrucdata-vg.sh \ testsuites/mmpstrucdata.conf \ + mmpstrucdata-invalid-vg.sh \ + testsuites/mmpstrucdata-invalid.conf \ libdbi-basic-vg.sh \ mmnormalize_variable.sh \ mmnormalize_tokenized.sh \ @@ -1366,6 +1374,14 @@ testsuites/stop_when_array_has_element.conf \ key_dereference_on_uninitialized_variable_space.sh \ testsuites/key_dereference_on_uninitialized_variable_space.conf \ + rscript_re_extract.sh \ + testsuites/rscript_re_extract.conf \ + rscript_re_match.sh \ + testsuites/rscript_re_match.conf \ + lookup_table.sh \ + testsuites/lookup_table.conf \ + testsuites/xlate.lkp_tbl \ + testsuites/xlate_more.lkp_tbl \ cfg.sh @@ -1385,7 +1401,7 @@ tcpflood_SOURCES = tcpflood.c tcpflood_CPPFLAGS = $(PTHREADS_CFLAGS) $(GNUTLS_CFLAGS) tcpflood_LDADD = $(SOL_LIBS) $(PTHREADS_LIBS) $(GNUTLS_LIBS) \ - $(am__append_24) + $(am__append_25) minitcpsrv_SOURCES = minitcpsrvr.c minitcpsrv_LDADD = $(SOL_LIBS) syslog_caller_SOURCES = syslog_caller.c @@ -1855,6 +1871,13 @@ --log-file $$b.log --trs-file $$b.trs \ $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ "$$tst" $(AM_TESTS_FD_REDIRECT) +rawmsg-after-pri.sh.log: rawmsg-after-pri.sh + @p='rawmsg-after-pri.sh'; \ + b='rawmsg-after-pri.sh'; \ + $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \ + --log-file $$b.log --trs-file $$b.trs \ + $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ + "$$tst" $(AM_TESTS_FD_REDIRECT) rfc5424parser.sh.log: rfc5424parser.sh @p='rfc5424parser.sh'; \ b='rfc5424parser.sh'; \ @@ -2499,6 +2522,41 @@ --log-file $$b.log --trs-file $$b.trs \ $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ "$$tst" $(AM_TESTS_FD_REDIRECT) +rscript_replace_complex.sh.log: rscript_replace_complex.sh + @p='rscript_replace_complex.sh'; \ + b='rscript_replace_complex.sh'; \ + $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \ + --log-file $$b.log --trs-file $$b.trs \ + $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ + "$$tst" $(AM_TESTS_FD_REDIRECT) +rscript_wrap2.sh.log: rscript_wrap2.sh + @p='rscript_wrap2.sh'; \ + b='rscript_wrap2.sh'; \ + $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \ + --log-file $$b.log --trs-file $$b.trs \ + $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ + "$$tst" $(AM_TESTS_FD_REDIRECT) +rscript_wrap3.sh.log: rscript_wrap3.sh + @p='rscript_wrap3.sh'; \ + b='rscript_wrap3.sh'; \ + $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \ + --log-file $$b.log --trs-file $$b.trs \ + $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ + "$$tst" $(AM_TESTS_FD_REDIRECT) +rscript_re_extract.sh.log: rscript_re_extract.sh + @p='rscript_re_extract.sh'; \ + b='rscript_re_extract.sh'; \ + $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \ + --log-file $$b.log --trs-file $$b.trs \ + $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ + "$$tst" $(AM_TESTS_FD_REDIRECT) +rscript_re_match.sh.log: rscript_re_match.sh + @p='rscript_re_match.sh'; \ + b='rscript_re_match.sh'; \ + $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \ + --log-file $$b.log --trs-file $$b.trs \ + $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ + "$$tst" $(AM_TESTS_FD_REDIRECT) rs_optimizer_pri.sh.log: rs_optimizer_pri.sh @p='rs_optimizer_pri.sh'; \ b='rs_optimizer_pri.sh'; \ @@ -2555,6 +2613,13 @@ --log-file $$b.log --trs-file $$b.trs \ $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ "$$tst" $(AM_TESTS_FD_REDIRECT) +lookup_table.sh.log: lookup_table.sh + @p='lookup_table.sh'; \ + b='lookup_table.sh'; \ + $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \ + --log-file $$b.log --trs-file $$b.trs \ + $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ + "$$tst" $(AM_TESTS_FD_REDIRECT) key_dereference_on_uninitialized_variable_space.sh.log: key_dereference_on_uninitialized_variable_space.sh @p='key_dereference_on_uninitialized_variable_space.sh'; \ b='key_dereference_on_uninitialized_variable_space.sh'; \ @@ -2757,27 +2822,6 @@ $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \ --log-file $$b.log --trs-file $$b.trs \ $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ - "$$tst" $(AM_TESTS_FD_REDIRECT) -rscript_replace_complex.sh.log: rscript_replace_complex.sh - @p='rscript_replace_complex.sh'; \ - b='rscript_replace_complex.sh'; \ - $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \ - --log-file $$b.log --trs-file $$b.trs \ - $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ - "$$tst" $(AM_TESTS_FD_REDIRECT) -rscript_wrap2.sh.log: rscript_wrap2.sh - @p='rscript_wrap2.sh'; \ - b='rscript_wrap2.sh'; \ - $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \ - --log-file $$b.log --trs-file $$b.trs \ - $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ - "$$tst" $(AM_TESTS_FD_REDIRECT) -rscript_wrap3.sh.log: rscript_wrap3.sh - @p='rscript_wrap3.sh'; \ - b='rscript_wrap3.sh'; \ - $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \ - --log-file $$b.log --trs-file $$b.trs \ - $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ "$$tst" $(AM_TESTS_FD_REDIRECT) imptcp_conndrop-vg.sh.log: imptcp_conndrop-vg.sh @p='imptcp_conndrop-vg.sh'; \ diff -Nru rsyslog-8.12.0/tests/mmpstrucdata-invalid-vg.sh rsyslog-8.14.0/tests/mmpstrucdata-invalid-vg.sh --- rsyslog-8.12.0/tests/mmpstrucdata-invalid-vg.sh 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/tests/mmpstrucdata-invalid-vg.sh 2015-10-23 12:52:03.000000000 +0200 @@ -0,0 +1,21 @@ +#!/bin/bash +# the goal here is to detect memleaks when structured data is not +# correctly parsed. +# This file is part of the rsyslog project, released under ASL 2.0 +# rgerhards, 2015-04-30 +echo =============================================================================== +echo \[mmpstrucdata-invalid.sh\]: testing mmpstrucdata with invalid SD +. $srcdir/diag.sh init +. $srcdir/diag.sh startup-vg mmpstrucdata-invalid.conf +. $srcdir/diag.sh wait-startup +# we use different message counts as this hopefully aids us +# in finding which sample is leaking. For this, check the number +# of blocks lost and see what set they match. +. $srcdir/diag.sh tcpflood -m100 -M "\"<161>1 2003-03-01T01:00:00.000Z mymachine.example.com tcpflood - tag [tcpflood@32473 MSGNUM] invalid structured data!\"" +. $srcdir/diag.sh tcpflood -m200 -M "\"<161>1 2003-03-01T01:00:00.000Z mymachine.example.com tcpflood - tag [tcpflood@32473 MSGNUM ] invalid structured data!\"" +. $srcdir/diag.sh tcpflood -m300 -M "\"<161>1 2003-03-01T01:00:00.000Z mymachine.example.com tcpflood - tag [tcpflood@32473 MSGNUM= ] invalid structured data!\"" +. $srcdir/diag.sh tcpflood -m400 -M "\"<161>1 2003-03-01T01:00:00.000Z mymachine.example.com tcpflood - tag [tcpflood@32473 = ] invalid structured data!\"" +. $srcdir/diag.sh shutdown-when-empty +. $srcdir/diag.sh wait-shutdown-vg +. $srcdir/diag.sh check-exit-vg +. $srcdir/diag.sh exit diff -Nru rsyslog-8.12.0/tests/mmpstrucdata-vg.sh rsyslog-8.14.0/tests/mmpstrucdata-vg.sh --- rsyslog-8.12.0/tests/mmpstrucdata-vg.sh 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/tests/mmpstrucdata-vg.sh 2015-10-23 12:52:03.000000000 +0200 @@ -0,0 +1,14 @@ +#!/bin/bash +# This file is part of the rsyslog project, released under ASL 2.0 +# rgerhards, 2013-11-22 +echo =============================================================================== +echo \[mmpstrucdata.sh\]: testing mmpstrucdata +. $srcdir/diag.sh init +. $srcdir/diag.sh startup-vg mmpstrucdata.conf +sleep 1 +. $srcdir/diag.sh tcpflood -m100 -y +. $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +. $srcdir/diag.sh wait-shutdown-vg +. $srcdir/diag.sh check-exit-vg +. $srcdir/diag.sh seq-check 0 99 +. $srcdir/diag.sh exit diff -Nru rsyslog-8.12.0/tests/rawmsg-after-pri.sh rsyslog-8.14.0/tests/rawmsg-after-pri.sh --- rsyslog-8.12.0/tests/rawmsg-after-pri.sh 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/tests/rawmsg-after-pri.sh 2015-11-03 08:15:58.000000000 +0100 @@ -0,0 +1,23 @@ +#!/bin/bash +# This file is part of the rsyslog project, released under ASL 2.0 +. $srcdir/diag.sh init +. $srcdir/diag.sh startup rawmsg-after-pri.conf +. $srcdir/diag.sh tcpflood -m1 -P 129 +. $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +. $srcdir/diag.sh wait-shutdown # and wait for it to terminate +echo "Mar 1 01:00:00 172.20.245.8 tag msgnum:00000000:" > rsyslog.out.compare +NUMLINES=$(grep -c "^Mar 1 01:00:00 172.20.245.8 tag msgnum:00000000:$" rsyslog.out.log 2>/dev/null) + +if [ -z $NUMLINES ]; then + echo "ERROR: output file seems not to exist" + ls -l rsyslog.out.log + cat rsyslog.out.log + . $srcdir/diag.sh error-exit 1 +else + if [ ! $NUMLINES -eq 1 ]; then + echo "ERROR: output format does not match expectation" + cat rsyslog.out.log + . $srcdir/diag.sh error-exit 1 + fi +fi +. $srcdir/diag.sh exit diff -Nru rsyslog-8.12.0/tests/rscript_re_extract.sh rsyslog-8.14.0/tests/rscript_re_extract.sh --- rsyslog-8.12.0/tests/rscript_re_extract.sh 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/tests/rscript_re_extract.sh 2015-11-03 08:10:58.000000000 +0100 @@ -0,0 +1,14 @@ +#!/bin/bash +# added 2015-09-29 by singh.janmejay +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_re_extract.sh\]: test re_extract rscript-fn +. $srcdir/diag.sh init +. $srcdir/diag.sh startup rscript_re_extract.conf +. $srcdir/diag.sh tcpflood -m 1 -I $srcdir/testsuites/date_time_msg +echo doing shutdown +. $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +. $srcdir/diag.sh wait-shutdown +. $srcdir/diag.sh content-check "*Number is 19597*" +. $srcdir/diag.sh exit diff -Nru rsyslog-8.12.0/tests/rscript_re_match.sh rsyslog-8.14.0/tests/rscript_re_match.sh --- rsyslog-8.12.0/tests/rscript_re_match.sh 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/tests/rscript_re_match.sh 2015-11-03 08:10:58.000000000 +0100 @@ -0,0 +1,14 @@ +#!/bin/bash +# added 2015-09-29 by singh.janmejay +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_re_match.sh\]: test re_match rscript-fn +. $srcdir/diag.sh init +. $srcdir/diag.sh startup rscript_re_match.conf +. $srcdir/diag.sh tcpflood -m 1 -I $srcdir/testsuites/date_time_msg +echo doing shutdown +. $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +. $srcdir/diag.sh wait-shutdown +. $srcdir/diag.sh content-check "*Matched*" +. $srcdir/diag.sh exit diff -Nru rsyslog-8.12.0/tests/tcpflood.c rsyslog-8.14.0/tests/tcpflood.c --- rsyslog-8.12.0/tests/tcpflood.c 2015-07-08 16:31:07.000000000 +0200 +++ rsyslog-8.14.0/tests/tcpflood.c 2015-10-23 12:52:03.000000000 +0200 @@ -398,7 +398,7 @@ } } else { if(bRandomizeExtraData) - edLen = ((long) rand() + extraDataLen) % extraDataLen + 1; + edLen = ((unsigned long) rand() + extraDataLen) % extraDataLen + 1; else edLen = extraDataLen; memset(extraData, 'X', edLen); diff -Nru rsyslog-8.12.0/tests/testsuites/gethostname.conf rsyslog-8.14.0/tests/testsuites/gethostname.conf --- rsyslog-8.12.0/tests/testsuites/gethostname.conf 2015-07-02 13:39:19.000000000 +0200 +++ rsyslog-8.14.0/tests/testsuites/gethostname.conf 2015-10-23 12:52:03.000000000 +0200 @@ -1,7 +1,7 @@ module(load="../plugins/imudp/.libs/imudp") $IncludeConfig diag-common.conf -module(load="../plugins/imptcp/.libs/imptcp") -input(type="imptcp" port="13514") +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") $template hostname,"%hostname%" local0.* ./HOSTNAME;hostname diff -Nru rsyslog-8.12.0/tests/testsuites/lookup_table.conf rsyslog-8.14.0/tests/testsuites/lookup_table.conf --- rsyslog-8.12.0/tests/testsuites/lookup_table.conf 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/tests/testsuites/lookup_table.conf 2015-11-03 08:15:16.000000000 +0100 @@ -0,0 +1,9 @@ +$IncludeConfig diag-common.conf + +lookup_table(name="xlate" file="xlate.lkp_tbl") + +template(name="outfmt" type="string" string="- %msg% %$.lkp%\n") + +set $.lkp = lookup("xlate", $msg); + +action(type="omfile" file="./rsyslog.out.log" template="outfmt") diff -Nru rsyslog-8.12.0/tests/testsuites/mmpstrucdata-invalid.conf rsyslog-8.14.0/tests/testsuites/mmpstrucdata-invalid.conf --- rsyslog-8.12.0/tests/testsuites/mmpstrucdata-invalid.conf 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/tests/testsuites/mmpstrucdata-invalid.conf 2015-10-23 12:52:03.000000000 +0200 @@ -0,0 +1,10 @@ +$IncludeConfig diag-common.conf + +module(load="../plugins/mmpstrucdata/.libs/mmpstrucdata") +module(load="../plugins/imtcp/.libs/imtcp") + +input(type="imtcp" port="13514") + +action(type="mmpstrucdata") +if $msg contains "msgnum" then + action(type="omfile" file="rsyslog.out.log") diff -Nru rsyslog-8.12.0/tests/testsuites/rawmsg-after-pri.conf rsyslog-8.14.0/tests/testsuites/rawmsg-after-pri.conf --- rsyslog-8.12.0/tests/testsuites/rawmsg-after-pri.conf 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/tests/testsuites/rawmsg-after-pri.conf 2015-11-03 08:15:58.000000000 +0100 @@ -0,0 +1,8 @@ +$IncludeConfig diag-common.conf + +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") + +template(type="string" name="outfmt" string="%rawmsg-after-pri%\n") +if $syslogfacility-text == "local0" then + action(type="omfile" file="rsyslog.out.log" template="outfmt") diff -Nru rsyslog-8.12.0/tests/testsuites/rscript_random.conf rsyslog-8.14.0/tests/testsuites/rscript_random.conf --- rsyslog-8.12.0/tests/testsuites/rscript_random.conf 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/tests/testsuites/rscript_random.conf 2015-10-23 12:52:03.000000000 +0200 @@ -1,8 +1,8 @@ $IncludeConfig diag-common.conf template(name="outfmt" type="string" string="%$.random_no%\n") -module(load="../plugins/imptcp/.libs/imptcp") -input(type="imptcp" port="13514") +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") set $.random_no = random(10); diff -Nru rsyslog-8.12.0/tests/testsuites/rscript_re_extract.conf rsyslog-8.14.0/tests/testsuites/rscript_re_extract.conf --- rsyslog-8.12.0/tests/testsuites/rscript_re_extract.conf 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/tests/testsuites/rscript_re_extract.conf 2015-11-03 08:10:58.000000000 +0100 @@ -0,0 +1,9 @@ +$IncludeConfig diag-common.conf +template(name="outfmt" type="string" string="*Number is %$.number%*\n") + +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") + +set $.number = re_extract($msg, '.* ([0-9]+)$', 0, 1, 'none'); + +action(type="omfile" file="./rsyslog.out.log" template="outfmt") diff -Nru rsyslog-8.12.0/tests/testsuites/rscript_re_match.conf rsyslog-8.14.0/tests/testsuites/rscript_re_match.conf --- rsyslog-8.12.0/tests/testsuites/rscript_re_match.conf 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/tests/testsuites/rscript_re_match.conf 2015-11-03 08:10:58.000000000 +0100 @@ -0,0 +1,10 @@ +$IncludeConfig diag-common.conf +template(name="outfmt" type="string" string="*Matched*\n") + +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") + +if (re_match($msg, '.* ([0-9]+)$')) then { + action(type="omfile" file="./rsyslog.out.log" template="outfmt") +} + diff -Nru rsyslog-8.12.0/tests/testsuites/rscript_replace_complex.conf rsyslog-8.14.0/tests/testsuites/rscript_replace_complex.conf --- rsyslog-8.12.0/tests/testsuites/rscript_replace_complex.conf 2015-07-08 16:31:07.000000000 +0200 +++ rsyslog-8.14.0/tests/testsuites/rscript_replace_complex.conf 2015-10-23 12:52:03.000000000 +0200 @@ -1,8 +1,8 @@ $IncludeConfig diag-common.conf template(name="outfmt" type="string" string="%$.replaced_msg%\n") -module(load="../plugins/imptcp/.libs/imptcp") -input(type="imptcp" port="13514") +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") set $.replaced_msg = replace($msg, "syslog", "rsyslog"); set $.replaced_msg = replace($.replaced_msg, "hello", "hello_world"); diff -Nru rsyslog-8.12.0/tests/testsuites/rscript_wrap2.conf rsyslog-8.14.0/tests/testsuites/rscript_wrap2.conf --- rsyslog-8.12.0/tests/testsuites/rscript_wrap2.conf 2015-07-08 16:31:07.000000000 +0200 +++ rsyslog-8.14.0/tests/testsuites/rscript_wrap2.conf 2015-10-23 12:52:03.000000000 +0200 @@ -1,8 +1,8 @@ $IncludeConfig diag-common.conf template(name="outfmt" type="string" string="%$.replaced_msg%\n") -module(load="../plugins/imptcp/.libs/imptcp") -input(type="imptcp" port="13514") +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") set $.replaced_msg = wrap("foo says" & $msg, "*" & "*"); diff -Nru rsyslog-8.12.0/tests/testsuites/rscript_wrap3.conf rsyslog-8.14.0/tests/testsuites/rscript_wrap3.conf --- rsyslog-8.12.0/tests/testsuites/rscript_wrap3.conf 2015-07-08 16:31:07.000000000 +0200 +++ rsyslog-8.14.0/tests/testsuites/rscript_wrap3.conf 2015-10-23 12:52:03.000000000 +0200 @@ -1,8 +1,8 @@ $IncludeConfig diag-common.conf template(name="outfmt" type="string" string="%$.replaced_msg%\n") -module(load="../plugins/imptcp/.libs/imptcp") -input(type="imptcp" port="13514") +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") set $.replaced_msg = wrap("foo says" & $msg, "bc" & "def" & "bc", "ES" & "C"); diff -Nru rsyslog-8.12.0/tests/testsuites/stop_when_array_has_element.conf rsyslog-8.14.0/tests/testsuites/stop_when_array_has_element.conf --- rsyslog-8.12.0/tests/testsuites/stop_when_array_has_element.conf 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/tests/testsuites/stop_when_array_has_element.conf 2015-10-23 12:52:03.000000000 +0200 @@ -2,8 +2,8 @@ template(name="foo" type="string" string="%$!foo%\n") module(load="../plugins/mmjsonparse/.libs/mmjsonparse") -module(load="../plugins/imptcp/.libs/imptcp") -input(type="imptcp" port="13514") +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") action(type="mmjsonparse") diff -Nru rsyslog-8.12.0/tests/testsuites/xlate.lkp_tbl rsyslog-8.14.0/tests/testsuites/xlate.lkp_tbl --- rsyslog-8.12.0/tests/testsuites/xlate.lkp_tbl 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/tests/testsuites/xlate.lkp_tbl 2015-11-03 08:15:16.000000000 +0100 @@ -0,0 +1,5 @@ +{ + "table":[ + {"index":" msgnum:00000000:", "value":"foo_old" }, + {"index":" msgnum:00000001:", "value":"bar_old" }] +} diff -Nru rsyslog-8.12.0/tests/testsuites/xlate_more.lkp_tbl rsyslog-8.14.0/tests/testsuites/xlate_more.lkp_tbl --- rsyslog-8.12.0/tests/testsuites/xlate_more.lkp_tbl 1970-01-01 01:00:00.000000000 +0100 +++ rsyslog-8.14.0/tests/testsuites/xlate_more.lkp_tbl 2015-11-03 08:15:16.000000000 +0100 @@ -0,0 +1,6 @@ +{ + "table":[ + {"index":" msgnum:00000000:", "value":"foo_new" }, + {"index":" msgnum:00000001:", "value":"bar_new" }, + {"index":" msgnum:00000002:", "value":"baz" }] +} diff -Nru rsyslog-8.12.0/tools/omfile.c rsyslog-8.14.0/tools/omfile.c --- rsyslog-8.12.0/tools/omfile.c 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/tools/omfile.c 2015-11-03 08:17:49.000000000 +0100 @@ -1037,14 +1037,16 @@ writeFile(pData, pParams, i); } /* Note: pStrm may be NULL if there was an error opening the stream */ - if(pData->bFlushOnTXEnd && pData->pStrm != NULL) { - /* if we have an async writer, it controls the flush via - * a timeout. However, without it, we actually need to flush, - * else incomplete records are written. - */ - if(!pData->bUseAsyncWriter) + if(pData->bUseAsyncWriter) { + if(pData->bFlushOnTXEnd && pData->pStrm != NULL) { CHKiRet(strm.Flush(pData->pStrm)); + } + } else { + if(pData->pStrm != NULL) { + CHKiRet(strm.Flush(pData->pStrm)); + } } + finalize_it: pthread_mutex_unlock(&pData->mutWrite); ENDcommitTransaction diff -Nru rsyslog-8.12.0/tools/omfwd.c rsyslog-8.14.0/tools/omfwd.c --- rsyslog-8.12.0/tools/omfwd.c 2015-08-10 12:24:49.000000000 +0200 +++ rsyslog-8.14.0/tools/omfwd.c 2015-10-23 12:52:03.000000000 +0200 @@ -86,6 +86,11 @@ char *port; int protocol; int iRebindInterval; /* rebind interval */ + sbool bKeepAlive; + int iKeepAliveIntvl; + int iKeepAliveProbes; + int iKeepAliveTime; + # define FORW_UDP 0 # define FORW_TCP 1 /* following fields for UDP-based delivery */ @@ -128,6 +133,10 @@ uchar *pszStrmDrvrAuthMode; /* authentication mode to use */ int iTCPRebindInterval; /* support for automatic re-binding (load balancers!). 0 - no rebind */ int iUDPRebindInterval; /* support for automatic re-binding (load balancers!). 0 - no rebind */ + int bKeepAlive; + int iKeepAliveIntvl; + int iKeepAliveProbes; + int iKeepAliveTime; permittedPeers_t *pPermPeers; } configSettings_t; static configSettings_t cs; @@ -154,6 +163,10 @@ { "compression.stream.flushontxend", eCmdHdlrBinary, 0 }, { "maxerrormessages", eCmdHdlrInt, 0 }, { "rebindinterval", eCmdHdlrInt, 0 }, + { "keepalive", eCmdHdlrBinary, 0 }, + { "keepalive.probes", eCmdHdlrPositiveInt, 0 }, + { "keepalive.time", eCmdHdlrPositiveInt, 0 }, + { "keepalive.interval", eCmdHdlrPositiveInt, 0 }, { "streamdriver", eCmdHdlrGetWord, 0 }, { "streamdrivermode", eCmdHdlrInt, 0 }, { "streamdriverauthmode", eCmdHdlrGetWord, 0 }, @@ -161,7 +174,7 @@ { "resendlastmsgonreconnect", eCmdHdlrBinary, 0 }, { "udp.sendtoall", eCmdHdlrBinary, 0 }, { "udp.senddelay", eCmdHdlrInt, 0 }, - { "template", eCmdHdlrGetWord, 0 }, + { "template", eCmdHdlrGetWord, 0 } }; static struct cnfparamblk actpblk = { CNFPARAMBLK_VERSION, @@ -691,6 +704,14 @@ /* params set, now connect */ CHKiRet(netstrm.Connect(pWrkrData->pNetstrm, glbl.GetDefPFFamily(), (uchar*)pData->port, (uchar*)pData->target)); + + /* set keep-alive if enabled */ + if(pData->bKeepAlive) { + CHKiRet(netstrm.SetKeepAliveProbes(pWrkrData->pNetstrm, pData->iKeepAliveProbes)); + CHKiRet(netstrm.SetKeepAliveIntvl(pWrkrData->pNetstrm, pData->iKeepAliveIntvl)); + CHKiRet(netstrm.SetKeepAliveTime(pWrkrData->pNetstrm, pData->iKeepAliveTime)); + CHKiRet(netstrm.EnableKeepAlive(pWrkrData->pNetstrm)); + } } finalize_it: @@ -917,6 +938,10 @@ pData->pszStrmDrvrAuthMode = NULL; pData->iStrmDrvrMode = 0; pData->iRebindInterval = 0; + pData->bKeepAlive = 0; + pData->iKeepAliveProbes = 0; + pData->iKeepAliveIntvl = 0; + pData->iKeepAliveTime = 0; pData->bResendLastOnRecon = 0; pData->bSendToAll = -1; /* unspecified */ pData->iUDPSendDelay = 0; @@ -993,6 +1018,14 @@ } } else if(!strcmp(actpblk.descr[i].name, "rebindinterval")) { pData->iRebindInterval = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "keepalive")) { + pData->bKeepAlive = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "keepaliveprobes")) { + pData->iKeepAliveProbes = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "keepaliveintvl")) { + pData->iKeepAliveIntvl = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "keepalivetime")) { + pData->iKeepAliveTime = (int) pvals[i].val.d.n; } else if(!strcmp(actpblk.descr[i].name, "streamdriver")) { pData->pszStrmDrvr = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(actpblk.descr[i].name, "streamdrivermode")) { @@ -1241,6 +1274,11 @@ pData->iRebindInterval = (pData->protocol == FORW_TCP) ? cs.iTCPRebindInterval : cs.iUDPRebindInterval; + pData->bKeepAlive = cs.bKeepAlive; + pData->iKeepAliveProbes = cs.iKeepAliveProbes; + pData->iKeepAliveIntvl = cs.iKeepAliveIntvl; + pData->iKeepAliveTime = cs.iKeepAliveTime; + /* process template */ CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, getDfltTpl())); @@ -1311,6 +1349,10 @@ cs.bResendLastOnRecon = 0; cs.iUDPRebindInterval = 0; cs.iTCPRebindInterval = 0; + cs.bKeepAlive = 0; + cs.iKeepAliveProbes = 0; + cs.iKeepAliveIntvl = 0; + cs.iKeepAliveTime = 0; return RS_RET_OK; } @@ -1328,6 +1370,10 @@ CHKiRet(regCfSysLineHdlr((uchar *)"actionforwarddefaulttemplate", 0, eCmdHdlrGetWord, setLegacyDfltTpl, NULL, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"actionsendtcprebindinterval", 0, eCmdHdlrInt, NULL, &cs.iTCPRebindInterval, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"actionsendudprebindinterval", 0, eCmdHdlrInt, NULL, &cs.iUDPRebindInterval, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionsendtcpkeepalive", 0, eCmdHdlrBinary, NULL, &cs.bKeepAlive, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionsendtcpkeepalive_probes", 0, eCmdHdlrInt, NULL, &cs.iKeepAliveProbes, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionsendtcpkeepalive_intvl", 0, eCmdHdlrInt, NULL, &cs.iKeepAliveIntvl, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"actionsendtcpkeepalive_time", 0, eCmdHdlrInt, NULL, &cs.iKeepAliveTime, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"actionsendstreamdriver", 0, eCmdHdlrGetWord, NULL, &cs.pszStrmDrvr, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"actionsendstreamdrivermode", 0, eCmdHdlrInt, NULL, &cs.iStrmDrvrMode, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"actionsendstreamdriverauthmode", 0, eCmdHdlrGetWord, NULL, &cs.pszStrmDrvrAuthMode, NULL)); diff -Nru rsyslog-8.12.0/tools/rscryutil.1 rsyslog-8.14.0/tools/rscryutil.1 --- rsyslog-8.12.0/tools/rscryutil.1 2015-08-10 12:28:17.000000000 +0200 +++ rsyslog-8.14.0/tools/rscryutil.1 1970-01-01 01:00:00.000000000 +0100 @@ -1,205 +0,0 @@ -.\" Man page generated from reStructuredText. -. -.TH RSCRYUTIL 1 "2013-04-15" "" "" -.SH NAME -rscryutil \- Manage Encrypted Log Files -. -.nr rst2man-indent-level 0 -. -.de1 rstReportMargin -\\$1 \\n[an-margin] -level \\n[rst2man-indent-level] -level margin: \\n[rst2man-indent\\n[rst2man-indent-level]] -- -\\n[rst2man-indent0] -\\n[rst2man-indent1] -\\n[rst2man-indent2] -.. -.de1 INDENT -.\" .rstReportMargin pre: -. RS \\$1 -. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin] -. nr rst2man-indent-level +1 -.\" .rstReportMargin post: -.. -.de UNINDENT -. RE -.\" indent \\n[an-margin] -.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]] -.nr rst2man-indent-level -1 -.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]] -.in \\n[rst2man-indent\\n[rst2man-indent-level]]u -.. -.SH SYNOPSIS -.INDENT 0.0 -.INDENT 3.5 -.sp -.nf -.ft C -rscryutil [OPTIONS] [FILE] ... -.ft P -.fi -.UNINDENT -.UNINDENT -.SH DESCRIPTION -.sp -This tool performs various operations on encrypted log files. -Most importantly, it provides the ability to decrypt them. -.SH OPTIONS -.INDENT 0.0 -.TP -.B \-d\fP,\fB \-\-decrypt -Select decryption mode. This is the default mode. -.TP -.BI \-W\fP,\fB \-\-write\-keyfile \ -Utility function to write a key to a keyfile. The key can be obtained -via any method. -.TP -.B \-v\fP,\fB \-\-verbose -Select verbose mode. -.TP -.B \-f\fP,\fB \-\-force -Forces operations that otherwise would fail. -.TP -.BI \-k\fP,\fB \-\-keyfile \ -Reads the key from . File _must_ contain the key, only, no headers -or other meta information. Keyfiles can be generated via the -\fI\-\-write\-keyfile\fP option. -.TP -.BI \-p\fP,\fB \-\-key\-program \ -In this mode, the key is provided by a so\-called "key program". This program -is executed and must return the key to (as well as some meta information) -via stdout. The core idea of key programs is that using this interface the -user can implement as complex (and secure) method to obtain keys as -desired, all without the need to make modifications to rsyslog. -.TP -.BI \-K\fP,\fB \-\-key \ -TESTING AID, NOT FOR PRODUCTION USE. This uses the KEY specified -on the command line. This is the actual key, and as such this mode -is highly insecure. However, it can be useful for intial testing -steps. This option may be removed in the future. -.TP -.BI \-a\fP,\fB \-\-algo \ -Sets the encryption algorightm (cipher) to be used. See below -for supported algorithms. The default is "AES128". -.TP -.BI \-m\fP,\fB \-\-mode \ -Sets the ciphermode to be used. See below for supported modes. -The default is "CBC". -.TP -.BI \-r\fP,\fB \-\-generate\-random\-key \ -Generates a random key of length . This option is -meant to be used together with \fI\-\-write\-keyfile\fP (and it is hard -to envision any other valid use for it). -.UNINDENT -.SH OPERATION MODES -.sp -The operation mode specifies what exactly the tool does with the provided -files. The default operation mode is "dump", but this may change in the future. -Thus, it is recommended to always set the operations mode explicitely. If -multiple operations mode are set on the command line, results are -unpredictable. -.SS decrypt -.sp -The provided log files are decrypted. Note that the \fI\&.encinfo\fP side files -must exist and be accessible in order for decryption to to work. -.SS write\-keyfile -.sp -In this mode no log files are processed; thus it is an error to specify -any on the command line. The specified keyfile is written. The key itself -is obtained via the usual key commands. If \fI\-\-keyfile\fP is used, that -file is effectively copied. -.sp -For security reasons, existing key files are _not_ overwritten. To permit -this, specify the \fI\-\-force\fP option. When doing so, keep in mind that lost -keys cannot be recovered and data encrypted with them may also be considered -lost. -.sp -Keyfiles are always created with 0400 permission, that is read access for only -the user. An exception is when an existing file is overwritten via the -\fI\-\-force\fP option, in which case the former permissions still apply. -.SH EXIT CODES -.sp -The command returns an exit code of 0 if everything went fine, and some -other code in case of failures. -.SH SUPPORTED ALGORITHMS -.sp -We basically support what libgcrypt supports. This is: -.INDENT 0.0 -.INDENT 3.5 -3DES -CAST5 -BLOWFISH -AES128 -AES192 -AES256 -TWOFISH -TWOFISH128 -ARCFOUR -DES -SERPENT128 -SERPENT192 -SERPENT256 -RFC2268_40 -SEED -CAMELLIA128 -CAMELLIA192 -CAMELLIA256 -.UNINDENT -.UNINDENT -.SH SUPPORTED CIPHER MODES -.sp -We basically support what libgcrypt supports. This is: -.INDENT 0.0 -.INDENT 3.5 -ECB -CFB -CBC -STREAM -OFB -CTR -AESWRAP -.UNINDENT -.UNINDENT -.SH EXAMPLES -.sp -\fBrscryutil logfile\fP -.sp -Decrypts "logfile" and sends data to stdout. -.sp -\fBrscryutil \-\-generate\-random\-key 16 \-\-keyfile /some/secured/path/keyfile\fP -.sp -Generates random key and stores it in the specified keyfile. -.SH LOG SIGNATURES -.sp -Encrypted log files can be used together with signing. To verify such a file, -it must be decrypted first, and the verification tool \fBrsgtutil(1)\fP must be -run on the decrypted file. -.SH SECURITY CONSIDERATIONS -.sp -Specifying keys directly on the command line (\fI\-\-key\fP option) is very -insecure and should -not be done, except for testing purposes with test keys. Even then it is -recommended to use keyfiles, which are also easy to handle during testing. -Keep in mind that command history is usally be kept by bash and can also -easily be monitored. -.sp -Local keyfiles are also a security risk. At a minimum, they should be -used with very restrictive file permissions. For this reason, -the \fIrscryutil\fP tool creates them with read permissions for the user, -only, no matter what umask is set to. -.sp -When selecting cipher algorithms and modes, care needs to be taken. The -defaults should be reasonable safe to use, but this tends to change over -time. Keep up with the most current crypto recommendations. -.SH SEE ALSO -.sp -\fBrsgtutil(1)\fP, \fBrsyslogd(8)\fP -.SH COPYRIGHT -.sp -This page is part of the \fIrsyslog\fP project, and is available under -LGPLv2. -.SH AUTHOR -Rainer Gerhards -.\" Generated by docutils manpage writer. -. diff -Nru rsyslog-8.12.0/tools/rscryutil.c rsyslog-8.14.0/tools/rscryutil.c --- rsyslog-8.12.0/tools/rscryutil.c 2015-07-02 13:39:19.000000000 +0200 +++ rsyslog-8.14.0/tools/rscryutil.c 2015-10-23 12:52:03.000000000 +0200 @@ -277,12 +277,22 @@ off64_t blkEnd; off64_t currOffs = 0; int r; + int fd; + struct stat buf; while(1) { /* process block */ if(initCrypt(eifp) != 0) goto done; - if((r = eiGetEND(eifp, &blkEnd)) != 0) goto done; + /* set blkEnd to size of logfp and proceed. */ + if((fd = fileno(logfp)) == -1) { + r = -1; + goto done; + } + if((r = fstat(fd, &buf)) != 0) goto done; + blkEnd = buf.st_size; + r = eiGetEND(eifp, &blkEnd); + if(r != 0 && r != 1) goto done; decryptBlock(logfp, outfp, blkEnd, &currOffs); gcry_cipher_close(gcry_chd); } diff -Nru rsyslog-8.12.0/tools/rsgtutil.1 rsyslog-8.14.0/tools/rsgtutil.1 --- rsyslog-8.12.0/tools/rsgtutil.1 2015-08-10 12:28:17.000000000 +0200 +++ rsyslog-8.14.0/tools/rsgtutil.1 2015-10-29 13:06:29.000000000 +0100 @@ -1,8 +1,8 @@ .\" Man page generated from reStructuredText. . -.TH RSCRYUTIL 1 "2013-04-15" "" "" +.TH RSGTUTIL 1 "2013-03-25" "" "" .SH NAME -rscryutil \- Manage Encrypted Log Files +rsgtutil \- Manage (GuardTime) Signed Log Files . .nr rst2man-indent-level 0 . @@ -36,61 +36,81 @@ .sp .nf .ft C -rscryutil [OPTIONS] [FILE] ... +rsgtutil [OPTIONS] [FILE] ... .ft P .fi .UNINDENT .UNINDENT .SH DESCRIPTION .sp -This tool performs various operations on encrypted log files. -Most importantly, it provides the ability to decrypt them. +This tool performs various maintenance operations on signed log files. +It specifically supports the GuardTime signature provider. +.sp +The \fIrsgtutil\fP tool is the primary tool to verify log file signatures, +dump signature file contents and carry out other maintenance operations. +The tool offers different operation modes, which are selected via +command line options. +.sp +The processing of multiple files is permitted. Depending on operation +mode, either the signature file or the base log file must be specified. +Within a single call, only a single operations mode is permitted. To +use different modes on different files, multiple calles, one for each +mode, must be made. +.sp +If no file is specified on the command line, stdin is used instead. Note +that not all operation modes support stdin. .SH OPTIONS .INDENT 0.0 .TP -.B \-d\fP,\fB \-\-decrypt -Select decryption mode. This is the default mode. +.B \-D\fP,\fB \-\-dump +Select "dump" operations mode. .TP -.BI \-W\fP,\fB \-\-write\-keyfile \ -Utility function to write a key to a keyfile. The key can be obtained -via any method. +.B \-t\fP,\fB \-\-verify +Select "verify" operations mode. .TP -.B \-v\fP,\fB \-\-verbose -Select verbose mode. +.B \-T\fP,\fB \-\-detect\-file\-type +Select "detect\-file\-type" operations mode. +.TP +.B \-B\fP,\fB \-\-show\-sigblock\-params +Select "show\-sigblock\-params" operations mode. +.TP +.B \-s\fP,\fB \-\-show\-verified +Prints out information about correctly verified blocks (by default, only +errors are printed). +.TP +.B \-e\fP,\fB \-\-extend +Select extend mode. This extends the RFC3161 signatures. Note that this +mode also implies a full verification. If there are verify errors, extending +will also fail. +.TP +.B \-c\fP,\fB \-\-convert +Select "conversion" mode. This converts signature files from +Version 10 to 11. The original file will automatically be backed up. .TP -.B \-f\fP,\fB \-\-force -Forces operations that otherwise would fail. +.B \-v\fP,\fB \-\-verbose +Select verbose mode. Most importantly, hashes and signatures are printed +in full length (can be \fBvery\fP lengthy) rather than the usual abbreviation. .TP -.BI \-k\fP,\fB \-\-keyfile \ -Reads the key from . File _must_ contain the key, only, no headers -or other meta information. Keyfiles can be generated via the -\fI\-\-write\-keyfile\fP option. -.TP -.BI \-p\fP,\fB \-\-key\-program \ -In this mode, the key is provided by a so\-called "key program". This program -is executed and must return the key to (as well as some meta information) -via stdout. The core idea of key programs is that using this interface the -user can implement as complex (and secure) method to obtain keys as -desired, all without the need to make modifications to rsyslog. -.TP -.BI \-K\fP,\fB \-\-key \ -TESTING AID, NOT FOR PRODUCTION USE. This uses the KEY specified -on the command line. This is the actual key, and as such this mode -is highly insecure. However, it can be useful for intial testing -steps. This option may be removed in the future. -.TP -.BI \-a\fP,\fB \-\-algo \ -Sets the encryption algorightm (cipher) to be used. See below -for supported algorithms. The default is "AES128". -.TP -.BI \-m\fP,\fB \-\-mode \ -Sets the ciphermode to be used. See below for supported modes. -The default is "CBC". -.TP -.BI \-r\fP,\fB \-\-generate\-random\-key \ -Generates a random key of length . This option is -meant to be used together with \fI\-\-write\-keyfile\fP (and it is hard -to envision any other valid use for it). +.BI \-P \ \fP,\fB \ \-\-publications\-server \ +Sets the publications server. If not set but required by the operation a +default server is used. The default server is not necessarily optimal +in regard to performance and reliability. +.TP +.B \-h\fP,\fB \-\-help +Shows short help for the utility. +.TP +.B \-d\fP,\fB \-\-debug +Enables additional debug output useful for developers. +.TP +.BI \-a \ \fP,\fB \ \-\-api \ +Specifies the API used by the utility. This parameter overwrites the Libary +used by this utility. However it is only possible to verify .gtsig signatures +with the old Guardtime library and .ksisig signatures only with the new +Guardtime KSI Libary. So the parameter won\(aqt have any affect when +verifying signatures. +Available options: +GT = Guardtime Client Library +KSI = Guardtime KSI Library .UNINDENT .SH OPERATION MODES .sp @@ -99,102 +119,79 @@ Thus, it is recommended to always set the operations mode explicitely. If multiple operations mode are set on the command line, results are unpredictable. -.SS decrypt +.SS dump .sp -The provided log files are decrypted. Note that the \fI\&.encinfo\fP side files -must exist and be accessible in order for decryption to to work. -.SS write\-keyfile -.sp -In this mode no log files are processed; thus it is an error to specify -any on the command line. The specified keyfile is written. The key itself -is obtained via the usual key commands. If \fI\-\-keyfile\fP is used, that -file is effectively copied. -.sp -For security reasons, existing key files are _not_ overwritten. To permit -this, specify the \fI\-\-force\fP option. When doing so, keep in mind that lost -keys cannot be recovered and data encrypted with them may also be considered -lost. -.sp -Keyfiles are always created with 0400 permission, that is read access for only -the user. An exception is when an existing file is overwritten via the -\fI\-\-force\fP option, in which case the former permissions still apply. +The provided \fIsignature\fP files are dumped. For each top\-level record, the*u +type code is printed as well as q short description. If there is additional +information available, it will be printed in tab\-indented lines below the +main record dump. The actual \fIlog\fP files need not to be present. +.SS verify +.sp +This mode does not work with stdin. On the command line, the \fIlog\fP file names +are specified. The corresponding \fIsignature\fP files (ending on ".gtsig") must also +be preset at the same location as the log file. In verify mode, both the log +and signature file is read and the validity of the log file checked. If verification +errors are detected these are printed and processing of the file aborted. By default, +each file is verified individually, without taking cross\-file hash chains into +account (so the order of files on the command line does not matter). +.sp +Note that the actual amount of what can be verified depends on the parameters with +which the signature file was written. If record and tree hashes are present, they +will be verified and thus fine\-granular error reporting is possible. If they are +not present, only the block signature itself is verified. +.sp +By default, only errors are printed. To also print successful verifications, use the +\fB\-\-show\-verified\fP option. +.SS extend +.sp +This extends the RFC3161 signatures. This includes a full verification +of the file. If there are verification errors, extending will also fail. +Note that a signature can only be extended when the required hash has been +published. Currently, these hashes are created at the 15th of each month at +0:00hrs UTC. It takes another few days to get them finally published. As such, +it can be assumed that extending is only possible after this happend (which +means it may take slightly above a month). +.sp +To prevent data corruption, a copy of the signature file is created during +extension. So there must be enough disk space available for both files, +otherwise the operation will fail. If the log file is named logfile, the +signature file is logfile.gtsig and the temporary work file is named +logfile.gtsig.new. When extending finished successfully, the original +signature file (logfile.gtsig in our example) is renamed with the .old +postfix (logfile.gtsig.old) and the temporary file written under the +original name. The .old file can be deleted. It is just kept as a +precaution to prevent signature loss. Note that any already existing +.old or .new files are overwritten by these operations. +.SS detect\-file\-type +.sp +This mode is used to detect the type of some well\-know files used inside the +signature system. The detection is based on the file header. This mode is +primarily a debug aid. +.SS show\-sigblock\-params +.sp +This mode is used to print signature block parameters. It is similar to \fIdump\fP +mode, but will ignore everything except signature blocks. Also, some additional +meta information is printed. This mode is primarily a debug aid. .SH EXIT CODES .sp The command returns an exit code of 0 if everything went fine, and some other code in case of failures. -.SH SUPPORTED ALGORITHMS -.sp -We basically support what libgcrypt supports. This is: -.INDENT 0.0 -.INDENT 3.5 -3DES -CAST5 -BLOWFISH -AES128 -AES192 -AES256 -TWOFISH -TWOFISH128 -ARCFOUR -DES -SERPENT128 -SERPENT192 -SERPENT256 -RFC2268_40 -SEED -CAMELLIA128 -CAMELLIA192 -CAMELLIA256 -.UNINDENT -.UNINDENT -.SH SUPPORTED CIPHER MODES -.sp -We basically support what libgcrypt supports. This is: -.INDENT 0.0 -.INDENT 3.5 -ECB -CFB -CBC -STREAM -OFB -CTR -AESWRAP -.UNINDENT -.UNINDENT .SH EXAMPLES .sp -\fBrscryutil logfile\fP -.sp -Decrypts "logfile" and sends data to stdout. +\fBrsgtutil \-\-verify logfile\fP .sp -\fBrscryutil \-\-generate\-random\-key 16 \-\-keyfile /some/secured/path/keyfile\fP +This verifies the file "logfile" via its associated signature file +"logfile.gtsig". If errors are detected, these are reported to stderr. +Otherwise, rsgtutil terminates without messages. .sp -Generates random key and stores it in the specified keyfile. -.SH LOG SIGNATURES +\fBrsgtutil \-\-dump logfile.gtsig\fP .sp -Encrypted log files can be used together with signing. To verify such a file, -it must be decrypted first, and the verification tool \fBrsgtutil(1)\fP must be -run on the decrypted file. -.SH SECURITY CONSIDERATIONS -.sp -Specifying keys directly on the command line (\fI\-\-key\fP option) is very -insecure and should -not be done, except for testing purposes with test keys. Even then it is -recommended to use keyfiles, which are also easy to handle during testing. -Keep in mind that command history is usally be kept by bash and can also -easily be monitored. -.sp -Local keyfiles are also a security risk. At a minimum, they should be -used with very restrictive file permissions. For this reason, -the \fIrscryutil\fP tool creates them with read permissions for the user, -only, no matter what umask is set to. -.sp -When selecting cipher algorithms and modes, care needs to be taken. The -defaults should be reasonable safe to use, but this tends to change over -time. Keep up with the most current crypto recommendations. +This dumps the content of the signature file "logfile.gtsig". The +actual log file is not being processed and does not even need to be +present. .SH SEE ALSO .sp -\fBrsgtutil(1)\fP, \fBrsyslogd(8)\fP +\fBrsyslogd(8)\fP .SH COPYRIGHT .sp This page is part of the \fIrsyslog\fP project, and is available under