Deleting the whole AtomSpace takes too long

[Welter]

[bug copied from Petaverse's bugzilla]

Deleting the AtomSpace object of an EmbodimentCogServer (specifically, an OPC) in its destructor takes too long.
The bigger the AtomSpace is, the longer it takes to be deleted (it seems a polynomial rate). So, this gets more critical after some time (several minutes) OPC is running. As an example, an AtomSpace that generates a dump file (using SavingLoading) of 5M bytes takes 25 seconds to be deleted. This may cause an error when someone unload a pet in a Virtual World and tries to reload it just after that (since the old OPC process will be stuck on AtomSpace deletion and will prevent a new OPC process to be started).

For now, since the AtomSpace is not released until cogserver process exits, it's not necessary to delete it. In fact, Cogserver just creates an instance of AtomSpace in its constructor but does not delete it in its destructor (probably because of this issue).

So, for now, AtomSpace deletion is disabled in OPC's destructor as well. This makes memory leaks to be detected by memory check tools (like valgrind). For this reason, there is special handling for this case in OPC's destructor using the CHECK_OPC_MEMORY_LEAKS configuration parameter.

[linas (linasvepstas)]

Please retest, this may be fixed already.

Approx half-a-year ago, I had made large changes to how AtomSpace stores indexes, in order to improve the deletion times of single atoms -- the deletion of just one atom was taking a very long time, (minutes!) when the atomspace got large.

I believe that these changes should dramatically improve atomspace deletion time, but have not tested.

[Welter (welterls)]

Hi Linas,

Actually, I've already spent some time re-testing this and rewrote the bug
description properly.
In fact, it seems it's faster now than at the time this bug was registered
(now it's taking several seconds, not minutes for a big (but not that big)
atomspace -- i.e., about 5Mbytes in a atomspace dump file). Anyway, this is
still a problem if AtomSpace is deleted when OPC exits (for the reason I
mentioned in bug description already) and it may also be a problem for
LearningServer, which discards an AtomSpace whenever a learning session is
done.
Also, this problem may be masked by the fact CogServer does not delete its
AtomSpace at the destructor.
You may try to add a delete atomSpace in CogServer destructor and check how
long it takes when a really large AtomSpace is in place (e.g., when WordNet
is loaded).

thanks,
Welter

2009/4/23 linas <email address hidden>

> Please retest, this may be fixed already.
>
> Approx half-a-year ago, I had made large changes to how AtomSpace stores
> indexes, in order to improve the deletion times of single atoms -- the
> deletion of just one atom was taking a very long time, (minutes!) when
> the atomspace got large.
>
> I believe that these changes should dramatically improve atomspace
> deletion time, but have not tested.
>
> --
> Deleting an AtomSpace object takes too long
> https://bugs.launchpad.net/bugs/365698
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in OpenCog Framework: New
>
> Bug description:
> [bug copied from Petaverse's bugzilla]
>
> Deleting the AtomSpace object of an EmbodimentCogServer (specifically, an
> OPC) in its destructor takes too long.
> The bigger the AtomSpace is, the longer it takes to be deleted (it seems a
> polynomial rate). So, this gets more critical after some time (several
> minutes) OPC is running. As an example, an AtomSpace that generates a dump
> file (using SavingLoading) of 5M bytes takes 25 seconds to be deleted. This
> may cause an error when someone unload a pet in a Virtual World and tries to
> reload it just after that (since the old OPC process will be stuck on
> AtomSpace deletion and will prevent a new OPC process to be started).
>
> For now, since the AtomSpace is not released until cogserver process exits,
> it's not necessary to delete it. In fact, Cogserver just creates an instance
> of AtomSpace in its constructor but does not delete it in its destructor
> (probably because of this issue).
>
> So, for now, AtomSpace deletion is disabled in OPC's destructor as well.
> This makes memory leaks to be detected by memory check tools (like
> valgrind). For this reason, there is special handling for this case in OPC's
> destructor using the CHECK_OPC_MEMORY_LEAKS configuration parameter.
>

[linas (linasvepstas)]

OK, yes. When the atomspace is deleted, it tries to delete all of the
atoms too. But instead of doing a bulk delete, I think it carefully removes each from each of the indexes. I think it can do this at a rate of 5K to 50K atoms/sec, so, yes, an atoms space with a few 100K
atoms will take too long. The trick is just to nuke the atoms, and never mind consistency of the indexes or sti/lti or anything else.