Request: export process-events and preserve I/O callbacks' order

[Derick Eddington]

Currently, in do-select, after ikrt_select returns, the pending list is iterated over again to see which of them select has said are ready. However, the new pending list (of those that weren't ready) which is created is left in reverse order (and then potentially reversed back and forth again and again); and when in-queue is used in process-events, it is reversed but shouldn't be. I think the order of pending needs to be preserved and in-queue not reversed so that the older registered callbacks get dispatched first. Otherwise, for a server like:

(let loop ()
  (let-values ([(to from)
                (accept-connection-nonblocking server-socket)])
    (register-callback to
      (lambda ()
        ---)))
  (loop))

which has many connections ready, the callbacks for the continuations of the accepts are dispatched before any of the lambda callbacks to handle a new client are, thereby starving these clients. Maybe this isn't the best example, but without preserving the order of callback registration in the order of dispatching, I worry unfair starvation situations will arise. Or am I missing something?

Attached is a tiny patch to preserve the order.

Also, a related question: what is the purpose of out-queue in process-events? Couldn't it be omitted and in-queue itself used?

[Derick Eddington (derick-eddington)]

Below are an example server and client I made to investigate this. For this server design, if there are connections always backlogged, starvation of clients really comes down to whether or not accept returns EAGAIN; if it does not, it won't capture its continuation and defer to process-events (I see long spurts of this happening with the below example); if none of the accepts return EAGAIN, it will just keep registering callbacks until it runs out of file-descriptors.

I think this is a good reason to both preserve callbacks order and to use a different design for servers: use register-callback to register server-sockets (already implemented in ikarus.io.ss) as well as client sockets and always defer to process-events, and re-register the server-socket after a new client's lambda is registered and then loop back to calling process-events, and with callbacks order preserved, clients' and the server's events will be serviced in the order they get registered, ensuring that, even if there are connections always backlogged, the events will be fairly serviced (I think). I think this will require exporting process-events. I think I'll make a test of this to see what happens...

;;;; Server ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
#!/usr/bin/env scheme-script
(import
  (rnrs)
  (only (ikarus)
        tcp-server-socket-nonblocking
        accept-connection-nonblocking
        register-callback
        printf))

(define listen-port 12345)
(define server-socket (tcp-server-socket-nonblocking listen-port))

(let loop ([i 0])
  (let-values ([(to from)
                (accept-connection-nonblocking server-socket)])
    (printf "Got new connection ~a from ~a\n" i to)
    (register-callback to
      (lambda ()
        (printf "Servicing connection ~a\n" i)
        (close-output-port to)
        (close-input-port from))))
  (loop (+ 1 i)))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

;;;; Client ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
#!/usr/bin/env scheme-script
(import
  (rnrs)
  (only (ikarus) tcp-connect))

(define server-address "127.0.0.1")
(define server-port "12345")

(let loop ()
  (let-values ([(to from) (tcp-connect server-address server-port)])
    (close-output-port to)
    (close-input-port from))
  (loop))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

[Derick Eddington (derick-eddington)]

With this new server below and the same client as above, the long spurts of accepting a ton of connections without servicing a single client do not happen. With my patch for preserving callbacks order, all the accepts of a new connection and servicing it are perfectly interleaved; and without this patch they're almost all interleaved but sometimes it'll accept a couple before servicing and then later service a couple before accepting a new. Unfortunately, this server is not as clean looking as the other, but it avoids the problem of relying on accept to return EAGAIN so that process-events happens; ways for making it cleaner would be cool.

On Sat, 2008-06-14 at 22:59 +0000, Derick Eddington wrote:
> and re-register the server-
> socket after a new client's lambda is registered and then loop back to
> calling process-events

JTM, looping back to call process-events again doesn't make sense of course because once process-events is entered, Ikarus never leaves it (unless you jump to a continuation you captured before).

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
#!/usr/bin/env scheme-script
(import
  (rnrs)
  (only (ikarus)
        tcp-server-socket-nonblocking
        accept-connection-nonblocking
        register-callback
        process-events
        printf))

(define listen-port 12345)
(define server-socket (tcp-server-socket-nonblocking listen-port))
(define i 0)

(define (accept-client)
  (let-values ([(to from)
                (accept-connection-nonblocking server-socket)])
    (printf "Got new connection ~a from ~a\n" i to)
    (register-callback to
      (let ([i i])
        (lambda ()
          (printf "Servicing connection ~a\n" i)
          (close-output-port to)
          (close-input-port from))))
    (register-callback server-socket accept-client)
    (set! i (+ 1 i))))

(register-callback server-socket accept-client)
(process-events)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

[Derick Eddington (derick-eddington)]

Aziz, do you have any problems with preserving the order of callback registration
in the order of dispatching? The necessity isn't as I at first thought, but it still seems fairer and I think it will help us reason better about a multiplexing server.

To avoid the accept possibly not returning EAGAIN issue, would you export process-events so that servers can be made like my example in my 3rd post?

I've also been wondering what is the purpose of out-queue in process-events? Could it be omitted and in-queue itself used?

Sorry some of my comments in the previous posts were too hasty.

On Fri, Jun 13, 2008 at 8:08 PM, Derick Eddington <email address hidden> wrote:
> I think the order of pending needs to be
> preserved and in-queue not reversed so that the older registered
> callbacks get dispatched first. Otherwise, for a server like:
> [...]
> which has many connections ready, the callbacks for the continuations of
> the accepts are dispatched before any of the lambda callbacks to handle
> a new client are, thereby starving these clients.

This is incorrect. The continuations of the accepts would not be dispatched before any of the callbacks to handle a new client. process-events dispatches all ready callbacks before doing do-select again.

On Sat, Jun 14, 2008 at 3:59 PM, Derick Eddington <email address hidden> wrote:
> if none of the accepts return EAGAIN, it will just keep
> registering callbacks until it runs out of file-descriptors.

It is more clear to say: if none of the accepts return EAGAIN, it will keep looping accepting connections until it runs out of file-descriptors.

[Abdulaziz Ghuloum (aghuloum)]

> Aziz, do you have any problems with preserving the order of callback registration in the order of dispatching?

No, not at all. I have been a little busy trying to workout the optimizer and couldn't give this enough thought. I'll look as soon as I get a chance. Sorry for the delay.

[Derick Eddington (derick-eddington)]

I suggest this be fixed for 0.0.4.