Review Board 1.7.22

RDMA iWarp protocol support

Review Request #128 - Created Nov. 30, 2010 and submitted

Andrew Stitcher
This is the combined set of changes to the RDMA protocol code to support iWarp and fix some bugs:

Following are the change logs for the individual changes:

*   Rename Rdma send buffer operations.

*   Changes due to review comments from Doug Ledford:
    - Removed lock unsafe operation Rdma::QueuePair::bufferAvailable()
      and replaced the unavailable case with failing getBuffer().
    - Improved asserts in the Rdma::QueuePair::getBuffer() code.

*   Implementation for v1 rdma protocol - append sent credit and flags word
    after sent frame (instead of sending it in immediate data).
*   Small change to send buffer management to support this to 0 dataCount
    when returning buffers.

*   Reduce rdma scatter gathers allocated per queue pair to 1 as iWarp
    only supports 1 sge.

*   Fix error handling for ibv_post_send() & ibv_port_recv()

*   Allow for the case where we get an rdma reject because there is no one listening
    on the resolved address

*   Factored rdma sending/receiving code out to make manipulating
    credit isolated

*   Reject attempts to connect with version of rdma protocol we don't understand

*   Plumb rdma protocol version into Rdma::AsynchIO

*   Add in some useful rdma warnings when we reject a connection

*   Add code to switch on/off byteswapping of RDMA ConnectionParams
    depending on detected protocol version. To give some backwards
    compatibility and fix a previous error which sent connection params
    in host order.

*   Added test into RdmaClient to be sure the messages we receive are the
    same as the message we sent. Use a pseudo random non-repetetive stream
    as the message to be sure there is no reordering or repeating of messages.

*   Allow RdmaClient to be interrupted
This has been tested with Mallanox IB cards and Chelsio & Intel Neteffects iWarp cards