SSL handshake failing with "sslv3 alert handshake failure:SSL alert number 40"

Question

I have a .pfx file, which can perfectly connect to the remote server when used on a windows client. I want to now connect to server using a linux client.

Problem 1) I used following openssl commands to extract public cert and private key from the pfx file,

openssl pkcs12 -in Name.pfx -nocerts -out priv.pem -nodes
openssl pkcs12 -in Name.pfx -nokeys -out pub.pem

But when I ran following two commands to verify md5 of both files, I found both of them different.

openssl rsa -noout -text -in priv.pem | openssl md5
openssl x509 -noout -text -in pub.pem | openssl md5

Problem 2) I instead used following command to extract single pem file from pfx that has both cert and key.

openssl pkcs12 -in Name.pfx -out bundle.pem

Using this pem file I tried connecting to the remote server, with following command :

openssl s_client -servername 1.2.3.4 -connect 1.2.3.4:1234 -CAfile bundle.pem -state -tls1_2

This gives following output on terminal

CONNECTED(00000003)
SSL_connect:before/connect initialization
SSL_connect:SSLv3 write client hello A
SSL_connect:SSLv3 read server hello A
depth=0 O = "My Name", CN = Name - Local
verify return:1
SSL_connect:SSLv3 read server certificate A
SSL_connect:SSLv3 read server key exchange A
SSL_connect:SSLv3 read server certificate request A
SSL_connect:SSLv3 read server done A
SSL_connect:SSLv3 write client certificate A
SSL_connect:SSLv3 write client key exchange A
SSL_connect:SSLv3 write change cipher spec A
SSL_connect:SSLv3 write finished A
SSL_connect:SSLv3 flush data
SSL3 alert read:fatal:handshake failure
SSL_connect:failed in SSLv3 read finished A
140250807310240:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure:s3_pkt.c:1275:SSL alert number 40
140250807310240:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:598:
---
Certificate chain
 0 s:/O=My Name /CN=Name - Local
   i:/O=My Name /CN=Name - Local
---
Server certificate
-----BEGIN CERTIFICATE-----
<random string of certificate>
-----END CERTIFICATE-----
subject=/O=My Name /CN=Name - Local
issuer=/O=My Name /CN=Name - Local
---
No client certificate CA names sent
Server Temp Key: ECDH, secp521r1, 521 bits
---
SSL handshake has read 1332 bytes and written 206 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: <some string>
    Session-ID-ctx: 
    Master-Key: <some string>
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    Start Time: 1495217834
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---

I cant figure out why is handshake failing. Stuck for 3 days, where exactly does problem lie.

Answer 1

But when I ran following two commands to verify md5 of both files, I found both of them different.

openssl rsa -noout -text -in priv.pem | openssl md5
openssl x509 -noout -text -in pub.pem | openssl md5

The first command shows textual information about the private key. The second textual information about the certificate which contains the public key. Of course these information are different.

Using this pem file I tried connecting to the remote server, with following command :

openssl s_client -servername 1.2.3.4 -connect 1.2.3.4:1234 -CAfile bundle.pem -state -tls1_2

This uses the certificate as a trusted CA (-CAfile). This is probably not what you want. Instead you want to use the certificate as a client certificate. This should be done as documented by using the options -cert and -key, i.e. -cert bundle.pem -key bundle.pem in your case.

Apart from that -servername should be a hostname and not an IP address. If you don't have a hostname skip this option.

SSL_connect:SSLv3 read server certificate request A
...
SSL_connect:SSLv3 write client certificate A
...
SSL3 alert read:fatal:handshake failure

Since you don't specify the client certificate properly an empty client certificate will be send. But the server expects a valid client certificate and thus report a failed handshake within an SSL alert back to the client.