System.Net.WebRequest and TLS 1.2 creates a 'Handshake Failure' with haproxy

Question

I'm unable to use System.Net.WebRequest for a request with TLS 1.2. If I do so, I get an Exception The request was aborted: Could not create SSL/TLS secure channel. and the protocol error Handshake Failure.

Connection over TLS 1.2 and authentication is working with Internet Explorer and Chrome. OpenSSL can connect to this enpoint over TLS 1.2.

Setup

1. Using ClientCertificates for authentication
2. Using non default Port
3. Using TLS 1.2
4. Endpoint is a haproxy but this is a blackbox
5. .NET 4.7 and C#

Code snipplet

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
var req = WebRequest.Create($"https://{host}:{port}");
((HttpWebRequest)req).ClientCertificates = new X509Certificate2Collection(GetCertificate());
var requestStream = req.GetRequestStream();

The method GetCertificate() returns a certificat with private key which is embedded in this software.

Wireshark

A text output from wireshark in the order the occure.

Client Hello

Secure Sockets Layer
    TLSv1.2 Record Layer: Handshake Protocol: Client Hello
        Content Type: Handshake (22)
        Version: TLS 1.2 (0x0303)
        Length: 207
        Handshake Protocol: Client Hello
            Handshake Type: Client Hello (1)
            Length: 203
            Version: TLS 1.2 (0x0303)
            Random: 5a292ab72d2173fc286aebe2c4cc991ee619e1cc81b5bb39...
            Session ID Length: 0
            Cipher Suites Length: 60
            Cipher Suites (30 suites)
            Compression Methods Length: 1
            Compression Methods (1 method)
            Extensions Length: 102
            Extension: server_name (len=43)
                Type: server_name (0)
                Length: 43
                Server Name Indication extension
                    Server Name list length: 41
                    Server Name Type: host_name (0)
                    Server Name length: 38
                    Server Name: [REMOVED]
            Extension: supported_groups (len=8)
                Type: supported_groups (10)
                Length: 8
                Supported Groups List Length: 6
                Supported Groups (3 groups)
            Extension: ec_point_formats (len=2)
                Type: ec_point_formats (11)
                Length: 2
                EC point formats Length: 1
                Elliptic curves point formats (1)
            Extension: signature_algorithms (len=20)
                Type: signature_algorithms (13)
                Length: 20
                Signature Hash Algorithms Length: 18
                Signature Hash Algorithms (9 algorithms)
            Extension: SessionTicket TLS (len=0)
                Type: SessionTicket TLS (35)
                Length: 0
                Data (0 bytes)
            Extension: extended_master_secret (len=0)
                Type: extended_master_secret (23)
                Length: 0
            Extension: renegotiation_info (len=1)
                Type: renegotiation_info (65281)
                Length: 1
                Renegotiation Info extension
                    Renegotiation info extension length: 0

Server Name: [REMOVED] contains the right server name.

Server Hello

Secure Sockets Layer
    TLSv1.2 Record Layer: Handshake Protocol: Server Hello
        Content Type: Handshake (22)
        Version: TLS 1.2 (0x0303)
        Length: 65
        Handshake Protocol: Server Hello
            Handshake Type: Server Hello (2)
            Length: 61
            Version: TLS 1.2 (0x0303)
            Random: 5a292ab7238205b2b8a2e6692abfd518a054515e53cd5b16...
            Session ID Length: 0
            Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
            Compression Method: null (0)
            Extensions Length: 21
            Extension: server_name (len=0)
                Type: server_name (0)
                Length: 0
            Extension: renegotiation_info (len=1)
                Type: renegotiation_info (65281)
                Length: 1
                Renegotiation Info extension
                    Renegotiation info extension length: 0
            Extension: ec_point_formats (len=4)
                Type: ec_point_formats (11)
                Length: 4
                EC point formats Length: 3
                Elliptic curves point formats (3)
            Extension: SessionTicket TLS (len=0)
                Type: SessionTicket TLS (35)
                Length: 0
                Data (0 bytes)

Certificate

Secure Sockets Layer
    TLSv1.2 Record Layer: Handshake Protocol: Certificate
        Content Type: Handshake (22)
        Version: TLS 1.2 (0x0303)
        Length: 3855
        Handshake Protocol: Certificate
            Handshake Type: Certificate (11)
            Length: 3851
            Certificates Length: 3848
            Certificates (3848 bytes)

Server Hello Done

Secure Sockets Layer
    TLSv1.2 Record Layer: Handshake Protocol: Server Key Exchange
        Content Type: Handshake (22)
        Version: TLS 1.2 (0x0303)
        Length: 589
        Handshake Protocol: Server Key Exchange
            Handshake Type: Server Key Exchange (12)
            Length: 585
            EC Diffie-Hellman Server Params
                Curve Type: named_curve (0x03)
                Named Curve: secp256r1 (0x0017)
                Pubkey Length: 65
                Pubkey: ...
                Signature Hash Algorithm: 0x0401
                Signature Length: 512
                Signature: ...

Multiple Handshake Messages

Secure Sockets Layer
    TLSv1.2 Record Layer: Handshake Protocol: Multiple Handshake Messages
        Content Type: Handshake (22)
        Version: TLS 1.2 (0x0303)
        Length: 77
        Handshake Protocol: Certificate
            Handshake Type: Certificate (11)
            Length: 3
            Certificates Length: 0
        Handshake Protocol: Client Key Exchange
            Handshake Type: Client Key Exchange (16)
            Length: 66
            EC Diffie-Hellman Client Params
                Pubkey Length: 65
                Pubkey: ...
    TLSv1.2 Record Layer: Change Cipher Spec Protocol: Change Cipher Spec
        Content Type: Change Cipher Spec (20)
        Version: TLS 1.2 (0x0303)
        Length: 1
        Change Cipher Spec Message
    TLSv1.2 Record Layer: Handshake Protocol: Encrypted Handshake Message
        Content Type: Handshake (22)
        Version: TLS 1.2 (0x0303)
        Length: 40
        Handshake Protocol: Encrypted Handshake Message

Handshake Failure

Secure Sockets Layer
    TLSv1.2 Record Layer: Alert (Level: Fatal, Description: Handshake Failure)
        Content Type: Alert (21)
        Version: TLS 1.2 (0x0303)
        Length: 2
        Alert Message
            Level: Fatal (2)
            Description: Handshake Failure (40)

Update

Thx to @user3484348 I have now more information.

TLS 1.2 (not working):

System.Net Information: 0 : [11752] InitializeSecurityContext(
credential = System.Net.SafeFreeCredential_SECURITY, 
context = 1054ea8:6091710, 
targetName = api.company.com, 
inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)

System.Net Information: 0 : [11752] InitializeSecurityContext(
In-Buffers count=2, 
Out-Buffer length=0, 
returned code=IllegalMessage)

TLS 1.0 (working):

System.Net Information: 0 : [11752] InitializeSecurityContext(
credential = System.Net.SafeFreeCredential_SECURITY, 
context = 12a5eb0:641d900, 
targetName = api.company.com, 
inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)

System.Net Information: 0 : [11752] InitializeSecurityContext(
In-Buffers count=2, 
Out-Buffer length=0, 
returned code=ContinueNeeded)

The returned code in TLS 1.2 is IllegalMessage and in TLS 1.0 it's ContinueNeeded.

Answer 1

Probably your GetCertificate() returns a certificate not from Windows Certificate repository. If you turn your application's System.Net debug info by adding

  <system.diagnostics>
<sources>
  <source name="System.Net">
    <listeners>
      <add name="System.Net"/>
    </listeners>
  </source>
  <source name="System.Net.Cache">
    <listeners>
      <add name="System.Net"/>
    </listeners>
  </source>
  <source name="System.Net.Http">
    <listeners>
      <add name="System.Net"/>
    </listeners>
  </source>
  <source name="System.Net.Sockets">
    <listeners>
      <add name="System.Net"/>
    </listeners>
  </source>
  <source name="System.Net.WebSockets">
    <listeners>
      <add name="System.Net"/>
    </listeners>
  </source>
</sources>
<switches>
  <add name="System.Net" value="Verbose"/>
  <add name="System.Net.Cache" value="Verbose"/>
  <add name="System.Net.Http" value="Verbose"/>
  <add name="System.Net.Sockets" value="Verbose"/>
  <add name="System.Net.WebSockets" value="Verbose"/>
</switches>
<sharedListeners>
  <add name="System.Net"
    type="System.Diagnostics.TextWriterTraceListener"
    initializeData="network.log"
  />
</sharedListeners>
<trace autoflush="true"/>

to your .config file, you will see "AcquireCredentialsHandle() failed with error 0X8009030D." or something similar. It seems System.Net cannot use a certificate with a private key if the key is not in the MachineKeys system folder.

Go the traditional route -- add the certificate to the repository, grant access to the key file, etc.