| 2014-06-19 03:05:44 |
Dmitry Kiselev |
description |
When I send 12000 buffers and RDMA read is triggered within the Accelio, but I get a CONNECTION_ERROR event instead of something meaningful.
The following block of code in xio_rdma_datapath.c, function: xio_sched_rdma_rd_req, obscures the error,
since EINVAL doesn't describe it, as well as the XIO_E_PARTIAL_MSG when no RDMA read is possible:
if (user_assign_flag) {
/* if user does not have buffers ignore */
if (task->imsg.in.data_iovlen == 0) {
WARN_LOG("application has not provided buffers\n");
WARN_LOG("rdma read is ignored\n");
task->imsg.status = XIO_E_PARTIAL_MSG;
return -1;
}
for (i = 0; i < task->imsg.in.data_iovlen; i++) {
if (task->imsg.in.data_iov[i].mr == NULL) {
ERROR_LOG("application has not provided mr\n");
ERROR_LOG("rdma read is ignored\n");
task->imsg.status = EINVAL;
return -1;
}
llen += task->imsg.in.data_iov[i].iov_len;
}
if (rlen > llen) {
ERROR_LOG("application provided too small iovec\n");
ERROR_LOG("remote peer want to write %zd bytes while" \
"local peer provided buffer size %zd bytes\n",
rlen, llen);
ERROR_LOG("rdma read is ignored\n");
task->imsg.status = EINVAL;
return -1;
}
Otherwise, please, kindly, let us know.
Also, from a surface look it looks as a kind of inconsistancy, when you can send messages even without registering a Memory Region up to a certain number of bytes, and then you add a byte and get a CONNECTION_ERROR.
Perhaps, if there is an edge there, the error reporting should be very presize, like ERR_MAX_LENGTH_WITH_NO_MR_IS_REACHED, or similar.
(Ideally it should be able to send all the message sizes (even sliced), or nothing at all)
Please, kindly, let me know if you had some other pattern in mind.
Thanks!
Dmitry |
When I send 12000 buffers and RDMA read is triggered within the Accelio, but I get a CONNECTION_ERROR event instead of something meaningful.
The following block of code in xio_rdma_datapath.c, function: xio_sched_rdma_rd_req, obscures the error,
since EINVAL doesn't describe it when no RDMA read is possible in my opinion:
if (user_assign_flag) {
/* if user does not have buffers ignore */
if (task->imsg.in.data_iovlen == 0) {
WARN_LOG("application has not provided buffers\n");
WARN_LOG("rdma read is ignored\n");
task->imsg.status = XIO_E_PARTIAL_MSG;
return -1;
}
for (i = 0; i < task->imsg.in.data_iovlen; i++) {
if (task->imsg.in.data_iov[i].mr == NULL) {
ERROR_LOG("application has not provided mr\n");
ERROR_LOG("rdma read is ignored\n");
task->imsg.status = EINVAL;
return -1;
}
llen += task->imsg.in.data_iov[i].iov_len;
}
if (rlen > llen) {
ERROR_LOG("application provided too small iovec\n");
ERROR_LOG("remote peer want to write %zd bytes while" \
"local peer provided buffer size %zd bytes\n",
rlen, llen);
ERROR_LOG("rdma read is ignored\n");
task->imsg.status = EINVAL;
return -1;
}
Otherwise, please, kindly, let us know.
Also, from a surface look it looks as a kind of inconsistancy, when you can send messages even without registering a Memory Region up to a certain number of bytes, and then you add a byte and get a CONNECTION_ERROR.
Perhaps, if there is an edge there, the error reporting should be very presize, like ERR_MAX_LENGTH_WITH_NO_MR_IS_REACHED, or similar.
(Ideally it should be able to send all the message sizes (even sliced), or nothing at all)
Please, kindly, let me know if you had some other pattern in mind.
Thanks!
Dmitry |
|
