Uploaded 'multipart/form-data' to ASP.NET Web API action, not readable in bufferless mode

Question

I'm building an ASP.NET Web API endpoint that accepts 'multipart/form-data' requests. I implemented it as described in this article using .NET Framework 4.5 and Web API 2.1. A simplified version of the action method I created, looks like this:

public async Task<HttpResponseMessage> PostFile()
{
    if (!Request.Content.IsMimeMultipartContent()) throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
    var rootPath = System.Configuration.ConfigurationManager.AppSettings["StorageLocation"].ToString();
    var provider = new MultipartFormDataStreamProvider(rootPath);
    var response = Request.CreateResponse(HttpStatusCode.OK);
    try
    {
        await Request.Content.ReadAsMultipartAsync(provider);
        // Imagine awesome logic here, unicorns and rainbows! Instead of that, we do the following:
        response.Content = new StringContent("You uploaded " + provider.FileData.Count.ToString() + " files.");
    }
    catch (Exception e) { throw new HttpResponseException(Request.CreateErrorResponse(HttpStatusCode.InternalServerError, e)); }
    return response;
}

Because the uploaded files can be very big (up to 2GiB), I want my requests to not be buffered by ASP.NET, thus avoiding high memory usage. To realize this I told Web API to stream incoming requests, instead of buffering them, as described in this article. The custom WebHostBufferPolicySelector looks something like this:

public class CustomWebHostBufferPolicySelector : WebHostBufferPolicySelector
{
    public override bool UseBufferedInputStream(object hostContext)
    {
        System.Web.HttpContextBase contextBase = hostContext as System.Web.HttpContextBase;
        if (contextBase != null && contextBase.Request.ContentType != null && contextBase.Request.ContentType.Contains("multipart")) return false;
        else return base.UseBufferedInputStream(hostContext);
    }

    public override bool UseBufferedOutputStream(System.Net.Http.HttpResponseMessage response)
    {
        return base.UseBufferedOutputStream(response);
    }
}

I load this guy in the Global.asax, at application start, like this:

protected void Application_Start(object sender, EventArgs e)
{
    // Here, other stuff got did.
    GlobalConfiguration.Configuration.Services.Replace(typeof(IHostBufferPolicySelector), new CustomWebHostBufferPolicySelector());
}

Alright, the board is set, lets get the pieces moving. If I don't use my CustomWebHostBufferPolicySelector, everything works just fine. However, when its used, I get the following exception:

Message: "An error has occurred."
ExceptionMessage: "Error reading MIME multipart body part."
ExceptionType: "System.IO.IOException"
StackTrace: " at System.Net.Http.HttpContentMultipartExtensions.<ReadAsMultipartAsync>d__0`1.MoveNext()\ \ --- End of stack trace from previous location where exception was thrown ---\ \ at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\ \ at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\ \ at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()\ \ at ..."

With the following inner exception:

Message: "An error has occurred."
ExceptionMessage: "Unable to read the entity body in Bufferless mode. The request stream has already been buffered."
ExceptionType: "System.InvalidOperationException"
StackTrace: " at System.Web.Http.WebHost.HttpControllerHandler.<>c__DisplayClass13.<GetStreamContent>b__10()\ \ at System.Web.Http.WebHost.HttpControllerHandler.LazyStreamContent.get_StreamContent()\ \ at System.Web.Http.WebHost.HttpControllerHandler.LazyStreamContent.CreateContentReadStreamAsync()\ \ at System.Net.Http.HttpContent.ReadAsStreamAsync()\ \ at System.Net.Http.HttpContentMultipartExtensions.<ReadAsMultipartAsync>d__0`1.MoveNext()"

It looks like the request is still buffered somehow, by something else. Is there another place in the ASP.NET pipeline I should be looking? Or even IIS maybe? What are the other places in this request's lifecycle where it can be buffered, and how do I control them?

Answer 1

In an attempt to make the problem more clear and shareable with others, I created a simple project to try and reproduce the problem. While doing this I found the answer: disable all kinds of tracing.

In my case I had ASP.NET's own tracing functionality enabled, and also Glimpse. Both of these buffer the request before it arrives at the Web API action.

For completeness' sake, here the proper way to turn them off in your Web.Config, while testing and in production.

<configuration>
  <system.web>
    <trace enabled="false" />
  </system.web>
  <glimpse defaultRuntimePolicy="Off">
  </glimpse>
</configuration>

In my case, these two were the culprits, but I can imagine there may be others, so be wary of this.