error processing archive (--unpack): unable to open '<FILE>.dpkg-new': Operation not permitted

[iczero]

dpkg has failed to install or upgrade certain packages recently, with the following (sample) output:

Unpacking linux-image-4.13.0-21-generic (4.13.0-21.24) over (4.13.0-21.24) ...
Preparing to unpack .../linux-image-extra-4.13.0-21-generic_4.13.0-21.24_amd64.deb ...
Unpacking linux-image-extra-4.13.0-21-generic (4.13.0-21.24) ...
dpkg: error processing archive /var/cache/apt/archives/linux-image-extra-4.13.0-21-generic_4.13.0-21.24_amd64.deb (--unpack):
 unable to open '/lib/modules/4.13.0-21-generic/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko.dpkg-new': Operation not permitted
[1] 18746 segmentation fault sudo apt install -f
                                                    %

This problem appears seemingly at random, and has already occurred several times. apt will sometimes segfault, but other times report that dpkg errored. Repeatedly running the command results in a different file being shown every time, seemingly sequentially. Removing the whole directory tree and retrying does not solve the problem.

Ubuntu version: 17.10
What was supposed to happen: linux-image-4.13.0-21-generic getting installed successfully
What happened instead: apt segfaulted after dpkg failed to write to a file

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Vitali Müller (e-ma5l-u)]

I can confirm this bug on a machine.
Ubuntu version: 17.10.

I tried to install texlive/kile etc.
The first error occured on "oxygen-icon-theme_5%3a5.38.0-0ubuntu1_all.deb", which has been reported already at https://askubuntu.com/questions/975837/apt-dpkg-error-ubuntu-17-10

After trying out several things, I could install oxygen, however, other packages were affected as well.

I do not think that this is a duplcate of bug #1685984, because this one is more general.

[Vitali Müller (e-ma5l-u)]

I think the problem was caused by sophos antivirus. After deactivating the on-demand scan with

sudo /opt/sophos-av/bin/savdctl disable

I had no problems.

[Radics László (garaboncias)]

I can confirm that sophos on-demand scan cause the problem
for me the error message was (but the file which is not able to open was random):
dpkg: error processing archive /var/cache/apt/archives/nodejs_8.14.0-1nodesource1_amd64.deb (--install):
 unable to open '/usr/lib/node_modules/npm/node_modules/tar/lib/mkdir.js.dpkg-new': Operation not permitted

[Ian (superian)]

This was happening on a machine here, interestingly with the module in linux-modules-extra-5.0.0-17 differing each time it was tried.

It does have Sophos AV installed on it, and disabling that does work.