~/.cache/upstart takes up around 70GB of space due to unity and mediascanner

[Owais Lone]

I just noticed that ~/.cache/upstart/ folder on my computer is taking up 69GB of space. Upon further investigation, I found out that mediascanner.log and unity-panel-service.log take 12GB and 54GB of space respectively.

I'm not entirely sure if this is related to upstart as I had removed logrotate package because of a conflict.

[James Hunt (jamesodhunt)]

This is not an Upstart issue:

- Upstart in Ubuntu provides a logrotate script that is run to compress session logs in ~/.cache/upstart/. So, by removing logrotate you have broken that facility.
- Upstart is correctly capturing the seemingly huge amount of data generated by these applications, so the problem lies with them.

However, in a sense your removing logrotate is good in that it has drawn attention to the problems with mediascanner and unity-panel-service.

I would encourage you to reinstall logrotate if possible.

[Colin Watson (cjwatson)]

If you had a conflict on logrotate, that suggests that you may be installing PostgreSQL packages built for a mismatching release of Ubuntu; due to a logrotate incompatibility they declare a Breaks on either old or new versions of logrotate depending on what release they're built for.

[Owais Lone (loneowais)]

Thanks for pointing that our Colin. Unfortunately, postgresql only publish packages for LTS releases. I've installed logrotate from 12.10 and held it with dpkg. Should work fine until an official solution.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Micool (micool-angers)]

I experienced the same, logrotate is installed and not conflicting, I had to delete a file around 60Go in size, the HDD was full (4ko remaining), and this happened 2 times sofar since I installed kubuntu 13.10.

the huge file was startkde.log. Next time it happens I will post a screenshot. I'm confident, it will ;(

Regards

[Gabriel (gnuetzi)]

Experienced the same problem in ubuntu 13.10

[dariocaruso (ing.gonzo)]

same on ubuntu 14.04

[Andrzej Golonka (wiecznywem)]

Same using Wine

[KFlash (kuv02)]

I just had the same problem, deleted the logfiles and noticed, that the log is running full in no time (several MB per Minute).

I opened the file and found this message over and over again:

> QGLContext::makeCurrent(): Cannot make invalid context current.

[ef (mike-globalfunkradio)]

Same in 14.04, never noticed it before on past ubuntu releases but this is first time i`m running on a small SSD where 70gb is a big deal

[ef (mike-globalfunkradio)]

So as I understand, Upstart is not at fault, and it's an application I am running, so I have left the ./config/upstart open in Nautilus, its's empty right now since deletion, will see what application initiates the problem, shabba

[ef (mike-globalfunkradio)]

Just happened again

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Oli (oli)]

This isn't limited to Unity. Just found my start-kde.log had swollen to several gigabytes because of a V4L bug that VLC had been spewing out for several minutes.

Upstart needs to be more clever about repeated messages. syslog can, for example, munch quick duplicates and say "Line repeated x times"

[Jeffrey Robbins (jeffreyrobbins)]

Just chiming in that this has effected me as well. Usually get the 'disk space is low' warning, then an 'rm -rf ~/.cache/upstart' clears up the issue. This has happened multiple times. Just tried removing logrotate and reinstalled as someone recommended earlier in the thread.

Running 14.04.1, base Ubuntu

[James Alred (801jesusstoner)]

confirming same issue, only thing that made it so bad was creating a autoban script for repeat offenders, in any event this is obviously a huge bug, just 'sudo rm -rf ~/.cache/upstart' and will baby sit this bad boy. Ubuntu 14.04.1 Desktop fresh install.... :(

[Diaa Sami (diaa.sami)]

Confirming on - update - Kubuntu 14.10 beta 2, didn't remove logrotate package, I just found that suddenly startkde.log got to 2.3 GB

[John Burkhart (jfburkhart)]

Confirming 14.04 .1 after recent update. Offending log: unity-panel-service.log

It instantly hits over 3 gb within minutes after deletion.

[Leo Milano (lmilano)]

Same here. My 120Gb SSD just filled up. Good thing I'm an advanced user and I could manage to find the offending folder. A newbie would just need to give up. The harddrive analyzer would not give any info (maybe because the disk was full).

In my case, it was unity7.log, which was about 50Gb.

Is there any way to completely stop the upstart logging? Shouldn't that be the default for regular users? This seems like a real bug.

Logrotate is installed here, and I'm running Ubuntu-14.10 (upgraded from older version for, well, close to 10 years now :-) ).

[maurus (maurus)]

Same problem, also unity7.log is the culprit.
It is filling with:

_xgeWireToEvent: Unknown extension 148, this should never happen.

I am running an upgraded 14.10.

[Leo Milano (lmilano)]

It just happened here. I will need to clean up and reboot. This is how it looks like

lmilano@grisell:upstart$ ls -lh unity7.log
-rw-r----- 1 lmilano users 53G Nov 12 16:03 unity7.log

lmilano@grisell:upstart$ tail unity7.log
_xgeWireToEvent: Unknown extension 148, this should never happen.
_xgeWireToEvent: Unknown extension 148, this should never happen.
_xgeWireToEvent: Unknown extension 148, this should never happen.
_xgeWireToEvent: Unknown extension 148, this should never happen.
_xgeWireToEvent: Unknown extension 148, this should never happen.
_xgeWireToEvent: Unknown extension 148, this should never happen.
_xgeWireToEvent: Unknown extension 148, this should never happen.
_xgeWireToEvent: Unknown extension 148, this should never happen.
_xgeWireToEvent: Unknown extension 148, this should never happen.
_xgeWireToEvent: Unknown extension 1lmilano@grisell:upstart$

[Leo Milano (lmilano)]

Ok, before rebooting, I just removed the offending file. It is growing at a rate of 1Gb every 10s or so. I will try logging out and in, and see if the log is still growing continuously ...

[Leo Milano (lmilano)]

To be clear: I removed the file before rebooting, but it got regenerated and refilled at a furious pace, with the same message: _xgeWireToEvent: Unknown extension 148, this should never happen.

At that time, "top" was reporting upstart as the most process, obviously caught in an infinite loop throwing this erroor about the extension 148.

It seems like this is the bug:
https://bugs.launchpad.net/ubuntu/+source/xorg/+bug/1369113

And it seems that the spamming, at least, would be removed with an upgrade of libXext to 1.3.3. Or at least, it would be nice to backport the relevant patch

http://patchwork.freedesktop.org/patch/22124/

[AndreK (andre-k)]

would somebody kill this stupid logevent soon ?
deletec the folder, purged/reinstalled logrotate, few minutes later 16GB log filling up disk again.
This waste laptop runtime - how to disable this freaking mess ?

[Chua Chee How (chuacheehow)]

I just want to report that I am also experiencing the same problem. The offending file is ~/.cache/upstart/gnome-session-Unity.log

It grows at a few megabytes every few seconds.

[Carles Oriol (carlesoriol)]

No news about this bug? Any solution=

[Stephen M. Webb (bregma)]

This is a largely misleading bug, at least as far as Unity is concerned.

The default set up is to rotate all user-session logs once every hour, with 7 previous logs saved compressed. With a normally operating desktop session, this should be more than good enough.

Where things go awry is if you get something like Xorg or an X11 client spamming the logs with messages. All application messages from programs running in the desktop session get logged to the desktop session log, generally called gnome-session-Unity.log or unity7.log, depending on the release of Ubuntu, or unity-panel-service.log for indicators. The misleading bit is that the vast majority of these spam messages are not in fact coming from Unity (which is relatively quiet) but from some application program you have chosen to run by clicking on a desktop icon or indicator you have installed.

In short, there is nothing Unity can do to reduce the amount of garbage being logged from third-party application programs. Unity has no control over what an arbitrary program outputs and does not have any access to the output streams from applications that it launches so it could add filters.

The only way to get bugs in arbitrary applications fixed is to file bugs against the broken applications explicitly. Filing a bug against the desktop shell that launches the applications will not bring much joy.

I'm marking this as invalid for Unity, since the problem does not originate in Unity nor is it possible to provide a fix in Unity for problems elsewhere. Bugs need to be filed against the individual applications causing the problem.

[Claudio Daniel Tenório de Barros (claudiodtbarros)]

Hello! I'm not a Ubuntu pro user, but I'm trying to learn more and more. Yesterday, I experienced the same situation as described here.

I work with molecular dynamics simulation, specifically with a program called GROMACS. When I was trying to do a simulation, the program told me it would take about 2 days to complete (weird, because the kind of simulation I had trying usually lasts about 6 hours). It only took a little time for my hard disk free space had being completely consumed (and breaking my simulation).

When I used the Baobap to see the disk usage, it took me to this folder .cache/upstart occupying about 70GB! I deleted it, restarted the computer, and my simulation had functioned normally (it took 5 hours to complete), and I didn't have any problems with the .cache. Very strange bug!!

[MasterCATZ (mastercatz)]

.cache/upstart/gnome-session-Unity.log

just started growing 9+ gig each day around 100 meg / minute ( just deleted and timed it )

it is just full off billions of lines of

#>

Ubuntu 14.04 LTS

[General-Beck (general-beck)]

/home/server/.cache/upstart/unity7.log.1 110 GB after update to 15.10
Ubuntu 15.10 x64

[Michi Henning (michihenning)]

mediascanner is no longer being maintained, marking as won't fix.

[Jelle De Loecker (skerit)]

It might not technically be unity's fault, why is unity logging a program's output anyway?

Spotify causes a similar issue with the `unity7.log` file, if you have too many files open it fills the log with this:

```
09:57:53.201 A [file_system_monitor_linux.cpp:104] Check failed: errno == EINTR:
09:57:53.201 A [file_system_monitor_linux.cpp:104] Check failed: errno == EINTR:
09:57:53.201 A [file_system_monitor_linux.cpp:104] Check failed: errno == EINTR:
```

Eating away gigabytes of space per minute, wasting write cycles of my SSD.

Maybe unity can skip adding identical lines, maybe it can notify the user when a program is putting out so much errors, maybe we can just disable the log? Anything but messing up our drive.

[ignik (igor-spb)]

# I will never read this trash:
ln -s /dev/null ~/.cache/upstart/unity7.log

[codywohlers (codywohlers)]

same issue with kodi spamming ~/.cache/upstart/startkde.log until there's no space left on /

kodi 2:17.0-git-20170210.1529-final-0trusty

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Vera Bucek (vbucek)]

I just had this happen to me. I discovered that the log file had reached a size of 315GB. I have a large drive so I didn't notice it until I went to log in and found that I couldn't. I'm running Linux Mint 18.1 Serena XFCE. The offending logfile is .cache/upstart/startxfce4.log. It's not growing that fast, but even though I would consider myself an intermediate Linux user, I have no idea how to analyze this logfile. So I'm hoping someone can help me figure out what the problem is. I've included the text from the log file below.
---------------

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): GVFS-WARNING **: can't init metadata tree /home/me/.local/share/gvfs-metadata/home: open: Permission denied

(Thunar:1729): thunar-WARNING **: Content type loading failed for startxfce4.log.1.gz: Error when getting information for file '/home/me/.cache/upstart/startxfce4.log.1.gz': No such file or directory

(Thunar:1729): GLib-GIO-CRITICAL **: g_file_info_get_attribute_uint32: assertion 'G_IS_FILE_INFO (info)' failed

(Thunar:1729): GLib-GIO-CRITICAL **: g_file_info_get_attribute_uint32: assertion 'G_IS_FILE_INF...

[psylem (subnetjet)]

Why not just disable this logging behaviour by default? No one knows about ~/.cache/upstart until it fills their disk and breaks their system (if they are savvy enough to find the problem log file).

The handful of power users that might occasionally need this logging behaviour can work out how to switch it on. In the mean time, not all of the users being impacted when it fills the disk will be able to work out what needs to be done about it.