HTTP Post using WebClient.UploadString is very slow when posting to another machine on same network

Question

we have a code below in a client app that posts data to an HTTP listener runs on another program.

try
{
  using (WebClient client = new WebClient())
  {
     client.Encoding = System.Text.Encoding.UTF8;
     client.Credentials = new NetworkCredential(NotificationUser, NotificationPassword);
     client.UploadString(NotificationUrl, msg);  // Notification URL is IP base not DNS name.
  }
}
catch (Exception ex){}

We are testing it in a high load environment and try to stress test the request/response latency. If I put both the programs on the same machine, I would get around 160 messages sent in one second from the posting app to the http listener app, but if I put the http listener app on different machine on the same network (a local network we create in house), that number goes down to about 5 messages/second.

Here are what we have tried so far:

1. Ping to the 2nd machine shows it responses in less then 1ms and tracert shows it has only one hop. No firewall or proxy between the two machines.
2. I used fiddler and StressStimulus to generate the heavy load of traffic to post to the listener app on another machine and I got (around 160 messages/second). In my opinion this rules out network latency or if the listener app is the problem.
3. I tried to use UploadStringAsync instead of UploadString in the posting appand that did not make much different.
4. No antivirus ect ...

The weird thing is this same code works normal if the listener app is on the same machine. Does anyone know that HTTP post has any restriction or something I overlook?

Answer 1

I found a question on here talking about WebClient() and HTTPWebRequest. So basically WebClient() is just a wrapper around httpwebRequest. We decided to test out my code by using HTTPWebRequest class instead.

try
{
    HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(Program.NotificationUrl);
    request.KeepAlive = false;
    request.Method = "Post";
    request.ContentType = "text/xml";
    request.Credentials = new System.Net.NetworkCredential(Program.NotificationUser, Program.NotificationPassword);

    byte[] data = Encoding.UTF8.GetBytes(msg);

    request.ContentLength = data.Length;
    Stream reqStream = request.GetRequestStream();
    reqStream.Write(data, 0, data.Length);
    reqStream.Close();

    WebResponse response = request.GetResponse();
    using (StreamReader reader = new StreamReader(response.GetResponseStream()))
    {
            reader.ReadToEnd();
    }

}
catch (Exception) 

Then what we found that really made the different was the flag KeepAlive. With a default value set to true, as soon as we set this flag to false, the whole http post process became lightning fast even with the authentication. If I was using WebClient class, this flag is not exposed to me and I assumed it kept the value KeepAlive=true by default.

Hopefully someone find this information useful down the road.