clamav-daemon (clamd) abends

[orange]

On Ubuntu 16.04.6 LTS
clamd will abend with nothing logged meaning mail will fail with a 451 error at the end of the DATA section.

Package clamav-daemon 0.100.2+dfsg-1ubuntu0.16.04.1 amd64

Enabling debug mode on clamd gives one the following:

Mar 01 11:38:39 mail clamd[7520]: clamd: yara_exec.c:177: yr_execute_code: Assertion `sp == 0' failed.
Mar 01 11:38:39 mail systemd[1]: clamav-daemon.service: Main process exited, code=killed, status=6/ABRT
Mar 01 11:38:39 mail clamsmtpd[492]: 10007F: clamd disconnected unexpectedly
Mar 01 11:38:39 mail systemd[1]: clamav-daemon.service: Unit entered failed state.
Mar 01 11:38:39 mail clamsmtpd[492]: 10007F: from=******, to=******
Mar 01 11:38:39 mail systemd[1]: clamav-daemon.service: Failed with result 'signal'.

Appears related to https://github.com/extremeshok/clamav-unofficial-sigs/issues/203
and the workaround suggested in that thread restores service.

[Maxim Britov (ungifted)]

Created attachment 7406
backtrace with debuginfo

I'm use clamav-unofficial-sigs.sh for additional clamav databases.
I found clamav-0.100.0-rc crashes with packer.yar and antidebug_antivm.yar
I will attach backtrace, clamd output, db examples.

Sat Apr 7 13:13:41 2018 -> Database correctly reloaded (13394577 signatures)
[New Thread 0x7fffbcb12700 (LWP 30615)]
clamd: yara_exec.c:177: yr_execute_code: Assertion `sp == 0' failed.

[Maxim Britov (ungifted)]

Created attachment 7407
clamd std log

[Maxim Britov (ungifted)]

Created attachment 7408
db packer.yar for example

[Micasnyd (micasnyd)]

Marking as public so it can be referenced in the clamav-users list.
In addition, we have an internal (Jira) task to follow up on this issue.

[Micasnyd (micasnyd)]

Another user reported crash due to clamav handling of yara rule sets.
Source:
 clamav-users: http://lists.clamav.net/pipermail/clamav-users/2018-May/006185.html

[Micasnyd (micasnyd)]

*** Bug 12117 has been marked as a duplicate of this bug. ***

[Jlbrown-u (jlbrown-u)]

Created attachment 7427
Clamscan crash log

[Jlbrown-u (jlbrown-u)]

I'm also using clamav-unofficial-sigs.sh, and I'm having clamscan crash.

crash log says:

Application Specific Information:
Assertion failed: (sp =3D=3D 0), function yr_execute_code, file =
yara_exec.c, line 177.

Have attached it.

[M-weissbach (m-weissbach)]

In my case, clamav is used for the mail traffic (amavis, postfix, dovecot) and the webproxy (squid over c-icap).
Hardware: Intel xeon, 24GB ECC-Ram, Raid5.
System: ArchLinux
After the update to clamav 0.100 it came to coredums and the accesses to the Internet over the squid-proxy became unbearably slow.
The following steps helped me to get the system working almost normally again. However, the system load on the CPU is considerably higher with clamav 0.100 than with clamav 0.99.4.
But at least I can now use the system again, without having to turn off the virus protection altogether. There are about 30 workstations on the system that use the mail server and the proxy.

Here are my steps:

1. Delete all signature databases that are located (in my case) under /var/lib/clamav.
2. Manual freshclam for the standard signatures
3. Setting default_dbs_rating="LOW" in /etc/clamav-unofficial-sigs/user.conf
4. Forcing reloading the signature databases with clamav-unofficial-sigs.sh -F

I use the following external sources:

sanesecurity_enabled = "yes"
securiteinfo_enabled = "yes"
linuxmalwaredetect_enabled = "yes"
malwarepatrol_enabled = "yes"
yararulesproject_enabled = "yes"
additional_enabled = "yes"

Maybe these steps will help you too.

[Micasnyd (micasnyd)]

*** Bug 12103 has been marked as a duplicate of this bug. ***

[Micasnyd (micasnyd)]

It seems to me that the assertion fail 'crash' when using antidebug_antivm.yar comes about after this commit:

https://github.com/Cisco-Talos/clamav-devel/commit/5891f83422e699f70e9f9bdcbcc9633f9a4cd5ef

Derived from:
https://bugzilla.clamav.net/show_bug.cgi?id=11567

I am guessing that what's going on is that before the change, it would abandon antidebug_antivm.yar rules when any of them failed to load, and that with the change, it only skips the ones that fail to load.

Before the change, I see:
LibClamAV Error: cli_loadyara: failed to parse rules file /Users/micasnyd/antidebug_antivm.yar, error count 7

With the change, I see:
LibClamAV Warning: cli_loadyara: failed to parse or load 7 yara rules from file /Users/micasnyd/antidebug_antivm.yar, successfully loaded 92 rules.

I haven't yet taken the time to identify which rules in antidebug_antivm.yar are failing, remove them, and verify if one of them still causes a crash in 0.99 and 0.100

[Bugreporter-j (bugreporter-j)]

I agree that previously it looks as if the files failed to compile and as a result were ignored with the annoying side effect of the error messages coming out on stderr and appearing in places they shouldn't.

I think that it's more than a little worrying that it's possible for uploaded .yar files to take out a running production clamd daemon.

I would like to be able to compile the code with NDEBUG off - so that the assertions in the code don't trigger the abort signal. I've never had to get to grips with configure - it seems to always define NDEBUG as on - even when --enable-debug is not included.

I tried --enable-debug=no which theoretically should do reverse the sense of the option, but to no avail.

There does seem to be considerable magic involved with the setting of debug flags in the compilation system.

[sergiomb (sergio-sergiomb)]

Hello ,
I have have this report https://bugzilla.redhat.com/show_bug.cgi?id=1590545 , and IMO we need a quick fix for clamav-0.100 ...

(In reply to Micah Snyder from comment #10)
(...)
> I haven't yet taken the time to identify which rules in antidebug_antivm.yar
> are failing, remove them, and verify if one of them still causes a crash in
> 0.99 and 0.100

for me crash just happens with 0.100 and just clamav-unofficial-sigs .
i.e , nobody complains about it in 0.99.[34]

[Micasnyd (micasnyd)]

My understanding (someone correct me if I'm wrong) is that the yara ruleset/database in question has been broken for a long time.

In 0.99.x some of the rules failed entirely, so the entire database was dropped. In 0.100, some of the rules failed, but it now allows it to partially load the ones that didn't outright fail. However, there appears to be a bug wherein at least one that is getting loaded is causing a crash.

Frankly, from my perspective this is an annoying bug that I want to fix, but it isn't a top priority because the yara ruleset in question didn't work with ClamAV from the get-go and therefore shouldn't have been published.

Using 3rd party signature databases is purely optional, so it isn't exactly a security flaw. I'd like to fix the yara parsing issues for 0.101, but I have no expectation of fixing it in an 0.100 patch release.

[Bugreporter-j (bugreporter-j)]

My solution to this was to remove the offending file from clamav-unofficial-sigs control file.

Change /etc/clamav-unofficial-sigs/master.conf to comment out the offending line:

#Antidebug_AntiVM/antidebug_antivm.yar|LOW # anti debug and anti virtualization techniques used by malware

and of course remove installed files. I had the advantage of compiling my own version of clamav and then could back up easily. This is not that easy of you are using yum packaged releases.

However, it is the case that if 100.0 is released for EPEL and used on systems where this file is installed - then clamd will just die.. and the ensuing complaint levels may not be good. Sadly it will look like a fault in clamd - and actually we are used to having faultless releases from you guys.

[sergiomb (sergio-sergiomb)]

I don't know you already know but just in case, antidebug_antivm.yar and EMAIL_Cryptowall.yar makes clamav-0.100 crashes

[1]
https://github.com/extremeshok/clamav-unofficial-sigs/issues/203

[Micksola (micksola)]

It wasn't quite clear at the offset of this bug, but ClamAV cannot support unofficial signatures from a development standpoint. For numerous reasons, we do not regress against those signatures, and in cases where sig writers publish non-functional signatures due to insufficient testing (which then cause crashes in newer versions of clam) we cannot devote our resources to fixing that problem.

We can only urge users to be more selective in which signature set they decide to trust, and ask sigwriters to push an update which removes the offending sigs.

All that said, we definitely encourage sigwriters to submit their signatures to undergo our official QA, signing, and distribution process. https://www.clamav.net/contact#partners

I don't want to dwell on "what could have beens", but if the writer of these sigs had taken advantage of our partner program, I imagine this problem would have been sussed out and fixed long ago.

Leaving this open for now, as we clearly have a bug in yara rule parsing. No promise on timeline. Please don't schedule around this issue.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Christian Ehrhardt  (paelzer)]

The security Team usually keeps clamav up to date throughout releases - so if that is an issue upstream it woudl be fixed on the next merge. But the issue linked is not part of the clamav upstream.

Furthermore the issue linked wants to disable yara rules which there seem to be none by default.
root@x:~# grep -Hrn yara /etc/*
root@x:~# ll /var/lib/clamav/
total 3
drwxr-xr-x 2 clamav clamav 2 Mar 4 15:22 ./
drwxr-xr-x 43 root root 43 Mar 4 15:22 ../

You said the workaround in that thread restored the service for you.

Might I ask:
1. where did you get the yara files - is that a package in the archive that I missed in my search?
2. since it doesn't trigger on a clean install of clamav can you provide the steps needed to trigger the issue?

[Christian Ehrhardt  (paelzer)]

Subscribing Marc (who did the recent uploads) but just as FYI