Read whole message with bufio.NewReader(conn)

Question

I am working on a simple chat server and client in golang. I am having some trouble with reading messages from the net.Conn. So far this is what I have been doing:

bufio.NewReader(conn).ReadString('\n')

Since the user presses enter to send the message I only have to read until '\n'. But I am now working on encryption and when sending the public keys between client and server the key sometimes contains '\n', which makes it hard to get the whole key. I am just wondering how I can read the whole message instead of stopping at a specific character. Thanks!

Answer 1

A simple option for sending binary data is to use a length prefix. Encode the data size as a 32bit big endian integer, then read that amount of data.

// create the length prefix
prefix := make([]byte, 4)
binary.BigEndian.PutUint32(prefix, uint32(len(message)))

// write the prefix and the data to the stream (checking errors)
_, err := conn.Write(prefix)
_, err = conn.Write(message)

And to read the message

// read the length prefix
prefix := make([]byte, 4)
_, err = io.ReadFull(conn, prefix)


length := binary.BigEndian.Uint32(prefix)
// verify length if there are restrictions

message = make([]byte, int(length))
_, err = io.ReadFull(conn, message)

See also Golang: TCP client/server data delimiter

You can also of course use an existing, well test protocol, like HTTP, IRC, etc. for your messaging needs. The go std library comes with a simple textproto package, or you could opt to enclose the messages in a uniform encoding, like JSON.