clamav-freshclam 1.0.5 and logwatch 7.7-1 fails to report

[heynnema]

$ lsb_release -rd
No LSB modules are available.
Description: Ubuntu 24.04 LTS
Release: 24.04

$ apt-cache policy clamav-freshclam
clamav-freshclam:
  Installed: 1.0.5+dfsg-1.1ubuntu3
  Candidate: 1.0.5+dfsg-1.1ubuntu3
  Version table:
 *** 1.0.5+dfsg-1.1ubuntu3 500
        500 http://us.archive.ubuntu.com/ubuntu noble/main amd64 Packages
        100 /var/lib/dpkg/status

$ apt-cache policy logwatch
logwatch:
  Installed: 7.7-1ubuntu1
  Candidate: 7.7-1ubuntu1
  Version table:
 *** 7.7-1ubuntu1 500
        500 http://us.archive.ubuntu.com/ubuntu noble/main amd64 Packages
        500 http://us.archive.ubuntu.com/ubuntu noble/main i386 Packages
        100 /var/lib/dpkg/status

Expect that daily logwatch emails properly show clamav update information.

I believe that this problem started around the time clamav went to version 1.0.x.

Excerpt from daily logwatch email report...

 --------------------- clam-update Begin ------------------------

 No updates detected in the log for the freshclam daemon (the
 ClamAV update process). If the freshclam daemon is not running,
 you may need to restart it. Other options:

 A. If you no longer wish to run freshclam, deleting the log file
    (configured is /var/log/clamav/freshclam.log ) will suppress this error message.

 B. If you use a different log file, update the appropriate
    configuration file. For example:
       echo "LogFile = log_file" >> /etc/logwatch/conf/logfiles/clam-update.conf
    where log_file is the filename of the freshclam log file.

 C. If you are logging using syslog, you need to indicate that your
    log file uses the syslog format. For example:
       echo "*OnlyService = freshclam" >> /etc/logwatch/conf/logfiles/clam-update.conf
       echo "*RemoveHeaders" >> /etc/logwatch/conf/logfiles/clam-update.conf

 ---------------------- clam-update End -------------------------

ProblemType: Bug
DistroRelease: Ubuntu 24.04
Package: clamav-freshclam 1.0.5+dfsg-1.1ubuntu3
ProcVersionSignature: Ubuntu 6.8.0-31.31-generic 6.8.1
Uname: Linux 6.8.0-31-generic x86_64
NonfreeKernelModules: nvidia_modeset nvidia
ApportVersion: 2.28.1-0ubuntu3
Architecture: amd64
CasperMD5CheckResult: pass
CurrentDesktop: ubuntu:GNOME
Date: Thu May 30 05:54:37 2024
InstallationDate: Installed on 2021-09-25 (978 days ago)
InstallationMedia: Ubuntu 21.04 "Hirsute Hippo" - Release amd64 (20210420)
ProcEnviron:
 LANG=en_US.UTF-8
 PATH=(custom, no user)
 SHELL=/bin/bash
 TERM=xterm-256color
 XDG_RUNTIME_DIR=<set>
SourcePackage: clamav
UpgradeStatus: Upgraded to noble on 2024-05-25 (5 days ago)

[Mitchell Dzurick (mitchdz)]

Thank you for making this bug report!

I wonder if this can be fixed by the solution in this blog[0], but the blog is pretty old so I'd be surprised such an old solution fixes it.

[0] - https://adriano.ws/fixing-updates-detected-log-freshclam-daemon-error/

[Mitchell Dzurick (mitchdz)]

[0] in my previous comment and still see the failure.

[Mitchell Dzurick (mitchdz)]

reproduction:

$ lxc launch ubuntu:noble n
$ lxc shell n
# apt update -y && apt install -y logwatch clamav
# logwatch --service clam-update

[Mitchell Dzurick (mitchdz)]

Actually I think what may be happening here is that clamav-freshclam.service updates the database routinely, so when logwatch attempts to update it, there is nothing to update, so the failure message is actually expected.

heynnema, how does your /var/log/clamav/freshclam.log look like now?

[heynnema (heynnema)]

@mitchdz You're partially correct.

clamav-freshclam.service updates the clam database, and updates the /var/log/clamav/freshclam.log.

logwatch does nothing to update "it". It only reviews the freshclam.log file, looking for certain markers, to indicate that updates occurred, or if there were any errors.

I had added the recent freshclam.log file as an attachment when I created the bug report. Please see it there.

I believe that either one of two things has happened...

1. Since clamav changed from a 0.98 version, to a 1.0.x version, the file format of freshclam.log has changed, and logwatch 7.x can no longer properly decipher it.

2. logwatch was updated to 7.7, from 7.5, and they didn't update the scripts for the clam-update portion.

Let me know if you need more.

[heynnema (heynnema)]

@mitchdz

You're missing one step in your comment #4.

A clam database update must occur at least once, for the freshclam.log to get populated.

systemctl status clamav-freshclam

sudo freshclam

[heynnema (heynnema)]

@mitchdz

The blog at https://adriano.ws/fixing-updates-detected-log-freshclam-daemon-error/ describes the exact situation... but it doesn't fix the problem... but the idea puts it into probably one of the correct areas to examine.

[heynnema (heynnema)]

Additional info added.

[NEOSYS Support (neosys)]

Don't know if this info will help fix the bug, but here goes.

The below log entry must exist in freshclam.log for logwatch to be able to decipher it:

"Fri Jul 12 11:38:56 2024 -> ClamAV update process started at Fri Jul 12 11:38:56 2024"

Tested this by manually editing and adding it to freshclam.log and then running logwatch.

freshclam.log :

Fri Jul 12 12:32:24 2024 -> --------------------------------------
Fri Jul 12 11:38:56 2024 -> ClamAV update process started at Fri Jul 12 11:38:56 2024
Fri Jul 12 12:32:24 2024 -> daily.cld database is up-to-date (version: 27334, sigs: 2064183, f-level: 90, builder: raynman)
Fri Jul 12 12:32:24 2024 -> main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
Fri Jul 12 12:32:24 2024 -> bytecode.cvd database is up-to-date (version: 335, sigs: 86, f-level: 90, builder: raynman)

Logwatch :

 --------------------- clam-update Begin ------------------------

 The following version(s) of the freshclam daemon were started
    0.103.11 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64): 1 Time(s)

 The ClamAV update process was started 1 time(s)

 Last ClamAV update process started at Fri Jul 12 11:38:56 2024

 Last Status:
    daily.cld database is up-to-date (version: 27334, sigs: 2064183, f-level: 90, builder: raynman)
    main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
    bytecode.cvd database is up-to-date (version: 335, sigs: 86, f-level: 90, builder: raynman)

 ---------------------- clam-update End -------------------------

[heynnema (heynnema)]

Thanks for the info. However, adding the log snippet to freshclam.log,
and then running logwatch, didn't provide any better results. Maybe
there's more that's needed.

It looks like you're running clamav 0.103.11, which is very old, but
precludes when I started having problems. I've always suspected that
some change between 0.103.11 and 1.0.5 caused the problem.

What version of clamav and logwatch are you running? What version of
Ubuntu? Any chance for you to upgrade to the latest clamav?

Cheers, Al

On 7/12/24 6:16 AM, NEOSYS Support wrote:
> Don't know if this info will help fix the bug, but here goes.
>
> The below log entry must exist in freshclam.log for logwatch to be able
> to decipher it:
>
> "Fri Jul 12 11:38:56 2024 -> ClamAV update process started at Fri Jul 12
> 11:38:56 2024"
>
> Tested this by manually editing and adding it to freshclam.log and then
> running logwatch.
>
> freshclam.log :
>
> Fri Jul 12 12:32:24 2024 -> --------------------------------------
> Fri Jul 12 11:38:56 2024 -> ClamAV update process started at Fri Jul 12 11:38:56 2024
> Fri Jul 12 12:32:24 2024 -> daily.cld database is up-to-date (version: 27334, sigs: 2064183, f-level: 90, builder: raynman)
> Fri Jul 12 12:32:24 2024 -> main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
> Fri Jul 12 12:32:24 2024 -> bytecode.cvd database is up-to-date (version: 335, sigs: 86, f-level: 90, builder: raynman)
>
>
> Logwatch :
>
> --------------------- clam-update Begin ------------------------
>
> The following version(s) of the freshclam daemon were started
> 0.103.11 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64): 1 Time(s)
>
> The ClamAV update process was started 1 time(s)
>
> Last ClamAV update process started at Fri Jul 12 11:38:56 2024
>
> Last Status:
> daily.cld database is up-to-date (version: 27334, sigs: 2064183, f-level: 90, builder: raynman)
> main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
> bytecode.cvd database is up-to-date (version: 335, sigs: 86, f-level: 90, builder: raynman)
>
> ---------------------- clam-update End -------------------------
>

--
*Al Heynneman*
<email address hidden>

[NEOSYS Support (neosys)]

logwatch --version
Logwatch 7.7 (released 07/22/22)

clamscan --version
ClamAV 1.0.5/27336/Sun Jul 14 08:33:25 2024

Ubuntu 24.04

The log snippet that I inserted (below line) was copied from an Ubuntu 22.04 server running ClamAV 0.103.11

"Fri Jul 12 11:38:56 2024 -> ClamAV update process started at Fri Jul 12 11:38:56 2024"

Looks like logwatch in my previous comment showed 0.103.11 because in one of the tests I did, I may have previously included a line "Mon Jul 15 06:28:54 2024 -> freshclam daemon 0.103.11 (OS: ....", before narrowing down to which line is actually required.

Replicated the bad "fix" again.

Steps:
1. nano /var/log/clamav/freshclam.log

2. Insert "Mon Jul 15 06:28:54 2024 -> ClamAV update process started at Mon Jul 15 06:28:54 2024" so your log file looks like:

Mon Jul 15 06:35:59 2024 -> --------------------------------------
Mon Jul 15 06:28:54 2024 -> ClamAV update process started at Mon Jul 15 06:28:54 2024
Mon Jul 15 06:35:59 2024 -> daily.cld database is up-to-date (version: 27336, sigs: 2064262, f-level: 90, builder: raynman)
Mon Jul 15 06:35:59 2024 -> main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
Mon Jul 15 06:35:59 2024 -> bytecode.cvd database is up-to-date (version: 335, sigs: 86, f-level: 90, builder: raynman)

3. Run logwatch with high detail
logwatch --detail high --range today | less

--------------------- clam-update Begin ------------------------

 The ClamAV update process was started 1 time(s)

 Last ClamAV update process started at Mon Jul 15 06:28:54 2024

 Last Status:
    daily.cld database is up-to-date (version: 27336, sigs: 2064262, f-level: 90, builder: raynman)
    main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
    bytecode.cvd database is up-to-date (version: 335, sigs: 86, f-level: 90, builder: raynman)

 ---------------------- clam-update End -------------------------

4. Extra test: Remove the "ClamAV update process started" line from freshclam.log and run logwatch again

logwatch --detail high --range today | less

 --------------------- clam-update Begin ------------------------

 No updates detected in the log for the freshclam daemon (the
 ClamAV update process). If the freshclam daemon is not running,
 you may need to restart it. Other options:

 A. If you no longer wish to run freshclam, deleting the log file
    (configured is /var/log/clamav/freshclam.log ) will suppress this error message.

 B. If you use a different log file, update the appropriate
    configuration file. For example:
       echo "LogFile = log_file" >> /etc/logwatch/conf/logfiles/clam-update.conf
    where log_file is the filename of the freshclam log file.

 C. If you are logging using syslog, you need to indicate that your
    log file uses the syslog format. For example:
       echo "*OnlyService = freshclam" >> /etc/logwatch/conf/logfiles/clam-update.conf
       echo "*RemoveHeaders" >> /etc/logwatch/conf/logfiles/clam-update.conf

 ---------------------- clam-update End -------------------------

[heynnema (heynnema)]

Thanks for the updated explanation. I'll play with it a bit more.

The line that you manually inserted doesn't appear to be generated in
1.0.5, so my belief that the problem began in clamav 1.x.x seems to be true.

I've got to comb through the 1.0.5 code to see why not.

Cheers, Al

On 7/14/24 11:53 PM, NEOSYS Support wrote:
> logwatch --version
> Logwatch 7.7 (released 07/22/22)
>
> clamscan --version
> ClamAV 1.0.5/27336/Sun Jul 14 08:33:25 2024
>
> Ubuntu 24.04
>
> The log snippet that I inserted (below line) was copied from an Ubuntu
> 22.04 server running ClamAV 0.103.11
>
> "Fri Jul 12 11:38:56 2024 -> ClamAV update process started at Fri Jul 12
> 11:38:56 2024"
>
> Looks like logwatch in my previous comment showed 0.103.11 because in
> one of the tests I did, I may have previously included a line "Mon Jul
> 15 06:28:54 2024 -> freshclam daemon 0.103.11 (OS: ....", before
> narrowing down to which line is actually required.
>
> Replicated the bad "fix" again.
>
> Steps:
> 1. nano /var/log/clamav/freshclam.log
>
> 2. Insert "Mon Jul 15 06:28:54 2024 -> ClamAV update process started at
> Mon Jul 15 06:28:54 2024" so your log file looks like:
>
> Mon Jul 15 06:35:59 2024 -> --------------------------------------
> Mon Jul 15 06:28:54 2024 -> ClamAV update process started at Mon Jul 15 06:28:54 2024
> Mon Jul 15 06:35:59 2024 -> daily.cld database is up-to-date (version: 27336, sigs: 2064262, f-level: 90, builder: raynman)
> Mon Jul 15 06:35:59 2024 -> main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
> Mon Jul 15 06:35:59 2024 -> bytecode.cvd database is up-to-date (version: 335, sigs: 86, f-level: 90, builder: raynman)
>
> 3. Run logwatch with high detail
> logwatch --detail high --range today | less
>
> --------------------- clam-update Begin ------------------------
>
> The ClamAV update process was started 1 time(s)
>
> Last ClamAV update process started at Mon Jul 15 06:28:54 2024
>
> Last Status:
> daily.cld database is up-to-date (version: 27336, sigs: 2064262, f-level: 90, builder: raynman)
> main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
> bytecode.cvd database is up-to-date (version: 335, sigs: 86, f-level: 90, builder: raynman)
>
> ---------------------- clam-update End -------------------------
>
>
> 4. Extra test: Remove the "ClamAV update process started" line from freshclam.log and run logwatch again
>
> logwatch --detail high --range today | less
>
> --------------------- clam-update Begin ------------------------
>
> No updates detected in the log for the freshclam daemon (the
> ClamAV update process). If the freshclam daemon is not running,
> you may need to restart it. Other options:
>
> A. If you no longer wish to run freshclam, deleting the log file
> (configured is /var/log/clamav/freshclam.log ) will suppress this error message.
>
> B. If you use a different log file, update the appropriate
> configuration file. For example:
> echo "LogFile = log_file" >> /etc/logwatch/conf/logfiles/clam-update.conf
> where log_file is the filename of the freshclam log file....

[Lucas Kanashiro (lucaskanashiro)]

If you check the logwatch source code and find the clamav script [1], you can see that this is the regex that identifies if the clamav daemon was started:

   } elsif (($ThisLine =~ /(?:Daemon started|clamd daemon [\d.]{1,10})/)) {
      $DaemonStart++;

Any string in the log that matches that will be counted as a start of the daemon.

Maybe there is a mismatch between the new clamav (1.x) and logwatch. If you find what exactly changed and the existing gap, an upstream bug should be filed.

Let us know if you make any advancement on that.

[1] https://sourceforge.net/p/logwatch/git/ci/7.7/tree/scripts/services/clamav