runc CVE-2019-5736 - check if kata is vulnerable

[Graham Whaley]

A CVE has been reported against `runc`. We should do due diligence and check if kata is affected or not, and take appropriate action.

"CVE-2019-5736: runc container breakout (all versions)"

https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/Tc1ELm-8oDI

[Graham Whaley (gwhaley)]

Chatted with EricE. I will ask JulioMontes if he can evaluate this for us, as he has experience in the area of libcontainer and our use in the agent.

[Julio Montes (devimc)]

Since exploit code will be release until 2019-02-18, I had to write an exploit in C to reproduce it and effectively, runC binary can be modified from inside the container :S.
As is mentioned in the report, the process that runs inside the container will be able to modify the binary until it's available (not running), in the case of kata-containers our "runC" binary is the kata-agent and in our case (fortunately) kata-agent is always busy (running), bu we should update libcontainers either way ;)

[Graham Whaley (gwhaley)]

Excellent Julio - thanks for reproducing and the clear diagnosis.
Are you able to PR an update to the libcontainers? Let me know when you have it ready and I will publish this report.

[Jeremy Stanley (fungi)]

When you do, it would also be a good idea to update the CVE-2019-5736 record with MITRE noting that Kata deployments aren't practically vulnerable to this.

[Graham Whaley (gwhaley)]

@fungi - great suggestion. Indeed, I think there are a few steps we need to take now:

1) Get one more pair of eyes to verify Julios findings
2) Post a kata status to the CVE record on MITRE
3) Email Aleksa (original reporter and runc author - I've met him before etc.) to let him know our findings.
4) Make this bug public
4) email kata-dev with an update

@Xu - could you cast your eyes over this CVE (or if you have somebody more suitable, nominate them?) to get a second confirmation of our findings please? thx.

[Graham Whaley (gwhaley)]

Update. I've emailed an early heads up and update to Aleksa privately. It may be that he has some deeper insight and ideas that are applicable to kata, so I'll be interested to see what he thinks.

[Peng Tao (bergwolf)]

FWIW, here is the attacking code [1] and a blog explaining the bug in more details by the original researcher [2].

[1]https://github.com/feexd/pocs/tree/master/CVE-2019-5736
[2]https://blog.dragonsector.pl/2019/02/cve-2019-5736-escape-from-docker-and.html

[Peng Tao (bergwolf)]

Right now since we do not restart kata-agent, we are not affected by the CVE. However, there were discussions about live-upgrading kata-agent in a running sandbox. So when that feature is implemented, we need to take care not be bitten by the same issue here.

Another concern raised by @lai is the memory footprint impact of the libcontainer fix[1]. It copies runc (in our case, kata-agent) binary to a separate place in memory for each new container and exec. And on my local machine, kata-agent is 22MB per se. It means we can easily eat up all guest memory if there are many execs.

Therefore, I would recommend holding up updating libcontainer dependency until we can find a better solution, or be ready to bite the bullet.

[1] https://github.com/opencontainers/runc/commit/0a8e4117e7f715d5fbeef398405813ce8e88558b

[Graham Whaley (gwhaley)]

I'm marking this item as 'medium' - we should do more investigation and update libcontainer, but there is no immediate threat to Kata Containers.

As per our VMT process (https://github.com/kata-containers/community/blob/master/VMT/VMT.md), I have filled out a KCSA document, and attached it here. Please **all** review this document. This will be a permanent record placed into the Kata github repository and posted publicly on our mailing list and slack channels.

I'd also like to discuss any public announcement. The intention will be to post the attached KCSA to the mailing lists and slack channels. I can post that, but I think I'd be more comfortable with the posting coming from a neutral address, possibly from an OSF address?

[Jeremy Stanley (fungi)]

Why will sending announcements from another address make you more comfortable? The only way Kata users and distributors are going to get to know and trust the community members handling vulnerability reports is to see them engaging in public activity like making official announcements. Signing the announcement cryptographically with your personal key is also a good way to provide attestation in the public record, but your users should ideally grow to trust Kata's community members who are doing this work. In most cases your users likely even have no idea who/what the OSF is so, seeing announcements from such an address conveys no useful authority for them anyway.

Other projects in the OSF family announce their vulnerability fixes through their community members as individuals. Even in my capacity as a vulnerability manager for OpenStack and Zuul I use my personal E-mail address for any announcements I send. In fact, I don't really use my OpenStack Foundation E-mail address for anything which isn't directly tied to actual foundation business, to make it clear I'm acting as just another regular member of the community.

[Eric (egernst)]

Agreed with Jeremy here. Thanks for the feedback, Jeremy.

[Archana Shinde (amshinde)]

This has long been resolved. Marking this as resolved to reflect so.