XDocument Load fails even though it can be accessed from chrome

Question

var doc = XDocument.Load("https://www.predictit.org/api/marketdata/all");

fails with the exception:

An unhandled exception of type 'System.Net.WebException' occurred in System.dll

Additional information: The underlying connection was closed: An unexpected error occurred on a send.

Even though you can access https://www.predictit.org/api/marketdata/all via chrome.

The api page: https://predictit.freshdesk.com/support/solutions/articles/12000001878-does-predictit-make-market-data-available-via-an-api- mentions appending text to the request header, may be a clue?

"To change the return type for any of the above, add one of the following to the request header.

Accept: application/xml

Accept: application/json"

I have checked my firewall, visual studio is allowed on both networks.

Answer 1

For that particular error, it's caused by a TLS handshake error. You can fix it by enabling the appropriate TLS protocols by adding this somewhere in your code:

System.Net.ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.Tls
    | System.Net.SecurityProtocolType.Tls11
    | System.Net.SecurityProtocolType.Tls12;

Once you have that sorted, you'll run into other problems with the response. You have to set the accept headers on the request. If you attempt to download without the headers, it will return JSON by default. The browser will include XML as one of the headers so that's why you're seeing XML.

You cannot do that using the basic XDocument.Load() overloads. You either have to download the contents separately as a string or at least obtain a stream with the correct headers and use the correct overload.

XDocument GetSecureXDocument(string url)
{
    System.Net.ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;
    var client = new WebClient
    {
        Headers = { [HttpRequestHeader.Accept] = "application/xml" }
    };
    using (var stream = client.OpenRead(url))
        return XDocument.Load(stream);
}