Logs are missing in /var/log/remote/node-X.test.domain.local

Bug #1523488 reported by Vladimir Khlyunev on 2015-12-07
24
This bug affects 4 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Critical
Ivan Suzdal

Bug Description

ISO 263
Snapshot https://product-ci.infra.mirantis.net/job/8.0.system_test.ubuntu.ha_neutron_destructive/65/artifact/logs/fail_error_ha_neutron_check_alive_rabbit-fuel-snapshot-2015-12-07_03-48-43.tar.xz

Steps
1) Deploy any cluster
2) Generate diag.snapshot and open it
or
2) Go to /var/log/remote/node-X.test.domain.local

Result:
Only puppet-apply log and bootstrap logs are presented.

This is blocker for debugging swarm jobs.

@bpiotrowski can help with that.

Changed in fuel:
status: New → Confirmed
Dmitry Teselkin (teselkin-d) wrote :
Changed in fuel:
assignee: Dmitry Teselkin (teselkin-d) → Ivan Suzdal (isuzdal)
Changed in fuel:
status: Confirmed → In Progress
Ivan Suzdal (isuzdal) wrote :

Seems like wrong fromhost regex in rsyslog
Before:
3433.140055678:7f6915822700: Filter: check for property 'fromhost' (value 'node-8.domain.tld') regex '^[0-9]': FALSE
After:
3490.784190932:7f6917ce7700: Filter: check for property 'fromhost' (value 'node-8.domain.tld') regex '^node-[0-9].*': TRUE

Ivan Suzdal (isuzdal) wrote :

There is request which should fix this issue
https://review.openstack.org/#/c/254867/

Bogdan Dobrelya (bogdando) wrote :

Marked as regression. The current match pattern have been working since the 3.1 release, so something get this broken in the 8.0

tags: added: regression-8.0
Bogdan Dobrelya (bogdando) wrote :

AFAICT, the /var/log/remote/node* destinations have been created by Fuel shotgun as symlinks. And real logs come from the IP addresses based sources.

tags: added: area-python
Changed in fuel:
assignee: Ivan Suzdal (isuzdal) → Dmitry Teselkin (teselkin-d)
Changed in fuel:
assignee: Dmitry Teselkin (teselkin-d) → Ivan Suzdal (isuzdal)

Reviewed: https://review.openstack.org/254867
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=d71c9b864bfd0931a01ed85fac743d8bad4c1744
Submitter: Jenkins
Branch: master

commit d71c9b864bfd0931a01ed85fac743d8bad4c1744
Author: Ivan Suzdal <email address hidden>
Date: Tue Dec 8 19:55:08 2015 +0300

    Use IP based regex for rsyslog

    From rsyslogd documentation:

      fromhost
        hostname of the system the message was received from
        (in a relay chain, this is the system immediately in
        front of us and not necessarily the original sender).
        This is a DNS-resolved name, except if that is not
        possible or DNS resolution has been disabled.

    So it might be either hostname or IP address, however,
    regex filter will work for IP address only.

    This commit forces IP usage of IP addresses only.

    Closes-Bug: #1523488

    Change-Id: I46e00f0273671a92126ce29a5bcebac257bedf03

Changed in fuel:
status: In Progress → Fix Released
Ivan Suzdal (isuzdal) on 2015-12-09
Changed in fuel:
status: Fix Released → Fix Committed

Used it in test and logs are present for all services.

Changed in fuel:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers