Server can't read as much data as he may need

[Paco]

Server side of jardin accepts a new connection, then reads as much as JARDIN_DEFAULT_RECV_BUFFER_SIZE and calls the user provided callback. Inside this callback if the caller needs to read more that or the data has not arrived yet, it isn't possible.

New method must be provided for user/caller to read as much as he needs.

[Aleksander Morgado (aleksander-m)]

Server is not really reading in the first iteration as much as JARDIN_DEFAULT_RECV_BUFFER_SIZE. The buffer may get filled only partially, and the user-provided callback gets called; then more bytes arrive, and user-provided callback gets called again. Note that this issue is only applicable to STREAM connections (TCP if INET), as with DGRAM the whole datagram arrives as a whole and user-provided callback is only called once.

If I understand the issue properly, you suggest to enable some way to wait & read more bytes inside the user-enabled callback right? If so, this is not really recommended, as you would be blocking the whole thread of execution while waiting for more data to come.

If waiting for a loong stream of bytes to arrive, it should be a task of the programmer to decide what to do while the whole stream didn't arrive. For example, if the stream of bytes is a 8GB DVD image arriving through TCP, the user-defined callback should never block & wait for the arrival of the whole chunk. In this case, the user-provided callback should receive each stream chunk in the recv buffer, and append the received bytes to a temporary file somewhere, until the whole stream of bytes arrives and the user-defined callback can properly generate the received file with all its contents.

If thinking in smaller streams of bytes, the temporary store with the already received stream chunks may be stored in heap memory instead of disk.

My 2 cents.

[Paco (manuel-arguelles)]

Yes,

I agree. But supposed we have a multi-thread jardin server instance, each connection will be placed on a thread, the user provided callback will be called in multiple threads.

In order for the caller to keep track of this, it has to share a memory area (maybe a hashtable with the recv_context.sfd as key (or the peer info)) to keep data coming from different peers, it also needs to know when the connection ends to remove the entry from the table.

I don't see any problem with blocking a thread... (as long as the block is not for ever), isn't that the whole purpose of using threads?

[Aleksander Morgado (aleksander-m)]

Per-thread data can be managed using GPrivate variables...
http://library.gnome.org/devel/glib/unstable/glib-Threads.html#GPrivate

Making a blocking-wait inside the user-defined callback could be hard to implement, but not impossible... and quite nasty regarding the API, as it would mean publishing the internal Jardin Connection inside the recv_context_s, and also publishing all Jardin Connection methods in the public Jardin API... or making a whole set of new methods on top of Jardin Server directly to manage the underlying Jardin Connection.

[Paco (manuel-arguelles)]

I will fix the client ;)