VMSF_DELTA filter in unrar allows arbitrary memory write

[Ralf Hildebrandt]

https://bugs.chromium.org/p/project-zero/issues/detail?id=1286&can=1&q=unrar&desc=6

It appears that the VMSF_DELTA memory corruption that was reported to Sophos AV in 2012 (and fixed there) was actually inherited from upstream unrar. For unknown reasons the information did not reach upstream rar or was otherwise lost, and the bug seems to have persisted there to this day.

The bug is sometimes called "Sophos Anti-Virus RAR VMSF_DELTA Filter Signedness Error", and referenced in the following links:
   https://community.sophos.com/kb/en-us/118424#six
   http://telussecuritylabs.com/threats/show/TSL20121207-01

The issue is in the following code:

    case VMSF_DELTA:
      {
        int DataSize=R[4],Channels=R[0],SrcPos=0,Border=DataSize*2;
        if ((uint)DataSize>=VM_MEMSIZE/2)
          break;

        // Bytes from same channels are grouped to continual data blocks,
        // so we need to place them back to their interleaving positions.
        for (int CurChannel=0;CurChannel<Channels;CurChannel++)
        {
          byte PrevByte=0;
          for (int DestPos=DataSize+CurChannel;DestPos<Border;DestPos+=Channels)
            Mem[DestPos]=(PrevByte-=Mem[SrcPos++]);
        }
      }
      break;

An attacker controls R[4] and R[0] arbitrarily. By setting R[0] to be sufficiently
large (perhaps 0x7FFFFFFF), an integer overflow can be caused in DataSize+CurChannel.
The result is a negative value of "DestPos", which allows the attacker to write
out of bounds when setting Mem[DestPos].

A minimal test example that crashes unrar 5.40 and prior is base64-encoded below:

Base64-encoded RAR file to trigger the VMSF_DELTA issue:

UmFyIRoHAPlOcwAADgAAAAAAAAAAMAh0AAAmAI4AAAAAAAAAAhBBUiEAAAAAHQAGAAAAACBzdGRv
dXQgIVUMzRDNmBGByDAda+AXaSv4KvQr1K/oejL05mXmXmww5tEk8gA9k8nmieyeyeswuOR6cx69
a2Hd6zQwu3aoMDDwMEswADAAMD4P938w+dydoRFwAmwAAAAAvv////+/////+9W3QFgAAQAGAAAA
Ooimhd12AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

Given the wide prevalence of the unrar source code in third-party software, quite a few
downstream parties will be affected. The source code with the fixes can be found
under http://www.rarlab.com/rar/unrarsrc-5.5.5.tar.gz - downstream parties are
encouraged to quickly update and ship fixes.

Thanks to the RAR developers for their quick turnaround and helpful proactive audit
of their codebase which removed further issues.

This bug is subject to a 90 day disclosure deadline. After 90 days elapse
or a patch has been made broadly available, the bug report will become
visible to the public.

[Tyler Hicks (tyhicks)]

Thanks for taking the time to report this bug and helping to make Ubuntu better. Since the package referred to in this bug is in universe or multiverse, it is community maintained. If you are able, I suggest coordinating with upstream and posting a debdiff for this issue. When a debdiff is available, members of the security team will review it and publish the package. See the following link for more information: https://wiki.ubuntu.com/SecurityTeam/UpdateProcedures

[Launchpad Janitor (janitor)]

[Expired for rar (Ubuntu) because there has been no activity for 60 days.]