RSA - How can the client (in server context) encrypt/decrypt incoming data?

Question

I have trouble understanding exactly how RSA works. Many examples and definitions attempt to explain but use a vague context.

So here is what I want to do:

I have a server that uses RSA and has a private and public key. From what I understand, upon connection the server sends the public key to the server in an unencrypted manner.. some kind of handshake if you will.

Further traffic would then occur in an encrypted fashion. To establish this fashion how would I be able to both encrypt what the client sends, and decrypt what the client receives (from client-sided perspective).

On server side encrypted data gets decrypted by the private key but I can't decrypt data in the client without that private key... that I can't have in the client because it's secret.

This confuses me, am I missing something obvious or is there need of a second private key?

Thanks in advance!

Answer 1

As you have already pointed out is RSA an asymmetric encryption scheme, that means:

c = E(pub_key, m) // ciphertext = encryption(public key,  message)
m = D(pri_key, c) // message    = decryption(private key, ciphertext)

In contrast a symmetric encryption scheme (such as fore example AES) works the following:

c = E(key, m)
m = E(key, c)

In other words the same key is used for encryption and decryption.

And that is where the asymmetric cryptosystem comes into play. It allows to parties to securely exchange a secret key for symmetric encryption.

---

So basically how a primitive (but very vulnerable!) key exchange could look:

1. server sends to client his public key pub_key_S
2. client sends to server his public key encrypted with the server's public key
   c = E(pub_key_S, pub_key_C)
3. server decrypts c with his private key pub_key_C = D(pri_key_S, c)
4. server generates a new random symmetric secret key key_CS
5. server encrypts the newly generated key with the client's public key c = E(pub_key_C, key_CS)
6. server sends c to the client
7. client decrypts ciphertext with his private key pri_key_C key_CS = D(pri_key_C, c)

Now Client and Server have a shared secret key key_CS which they can use to securely communicate for the ongoing session.

---

Such a protocol is in practice quite a bit more complicated, including certificates, digital signatures, hashcodes and so on. The probably most widely used protocol is SSL or TLS. (for in example https).

I recommend you to check out that link if you are interested in the details of such a protocol.