implementing sendall() and recvall() in C and python

Question

I'm currently trying to implement a sendall() function in a server written in C, and a recvall() function on the corresponding client written in python.

I can get the server and the client to work together when they're both written in the same language (i.e. both in c or both in python), but I can't get it to work with the server in c and the client in python. currently, i want to send a simple string from the c server to the python client.

Server sendall() implementation in C as follows (referenced from Beej's guide):

int sendall(int socket, char * buf, int *len){
    int total = 0;        // how many bytes we've sent
    int bytesleft = *len; // how many we have left to send
    int n;

    while(total < *len) {
        n = send(socket, buf + total, bytesleft, 0);
        if (n == -1) { break; }
        total += n;
        bytesleft -= n;
    }

    *len = total; // return number actually sent here

    return n==-1?-1:0; // return -1 on failure, 0 on success
}

Calling the function:

char buf[10] = "test";
    int len;

    len = strlen(buf);

    if (sendall(command_socket, buf, &len) == -1) {
        perror("sendall");
        printf("We only sent %d bytes because of the error!\n", len);
    }

    printf("Bytes sent: %d\n", len);

Client recvall() implementation in Python (referenced from http://stupidpythonideas.blogspot.com/2013/05/sockets-are-byte-streams-not-message.html):

def recv_one_message(sock):
    lengthbuf = recvall(sock, 4)
    length, = struct.unpack('!I', lengthbuf)
    return recvall(sock, length)

def recvall(sock, count):
    buf = ''
    while count:
        newbuf = sock.recv(count)
        print newbuf

        if not newbuf:
            return None

        buf += newbuf
        count -= len(newbuf)
    return buf

Called as:

buf = recv_one_message(command_socket)
print buf

Whenever I send a message from the C-server to the Python-client, I get a return of "None." I've traced the the response coming in on the client side -- it is getting the message I sent, but the final response is always "none" and the message won't be printed. I've also tried just returning the message instead of having the None return, which also results in nothing getting printed. Can't see where I'm going wrong, any thoughts? Thanks.

Answer 1

Say you have a perfectly fluent English speaker and a perfectly fluent French speaker. They will not be able to communicate with each other very well. Who is at fault? -- Whoever put the two of them together expecting them to be able to communicate without first agreeing on how they would communicate.

Your send function and your receive function implement totally different protocols. The send function requires the receiver to already know the length of the data to receive. Your receive function requires the sender to send the length prior to the data. So you cannot mix and match them because they do not use the same wire format

Here's some valuable advice gained over decades of experience: Never attempt to use TCP without first documenting the protocol you're going to use on top of TCP at the byte level. As it is, there's no way to know which function is right and which is wrong because nobody has any idea what bytes are supposed to be sent over the wire.

TCP is a byte-stream protocol, and for two programs to work correctly with a TCP connection between them, each must send precisely the stream of bytes the other expects to receive. The best way to make sure this happens is to first document that stream of bytes.

If you have no idea where to start with such documentation, have a look at the simplest existing protocol that's analogous to what you're trying to do. Possible protocols to look at it include HTTP, SMTP, IRC, and DNS.