Does FileStream.FlushAsync() ensure file is written to disk?

Question

On the FileStream class, there is a difference between calling Flush(false) and Flush(true), the latter will cause any buffered data to be written to the file.

However it is not clear what is the behavior of FlushAsync, as it doesn't accept a flushToDisk parameter.

I suppose my real question is, how can I ensure a file is written to disk, asynchronously?

Answer 1

Determining the actual write/flush behavior in .NET Core for disk streams requires a deep dive into the source code, because as pointed out by Jonathan the documentation here is unfortunately inaccurate (or at best highly misleading).

First the behavior depends on if the FileStream is opened with buffering (the default) or not. If there's no buffering, the behavior is defined in OSFileStreamStrategy. If it does buffer, it wraps an OSFileStreamStrategy in a BufferedFileStreamStrategy.

Starting with the less common but simpler case of no buffering, see the FlushAsync override:

public sealed override Task FlushAsync(CancellationToken cancellationToken) => 
       Task.CompletedTask; // no buffering = nothing to flush

It turns out FlushAsync is a no-op in the non-buffering case. The bytes have left the process and are in the hands of the OS now, but the OS has not been independently told to flush the bytes to disk.

Now note the very next lines in that file:

internal sealed override void Flush(bool flushToDisk)
{
    if (flushToDisk && CanWrite)
    {
        FileStreamHelpers.FlushToDisk(_fileHandle);
    }
}

So not only is FlushAsync always a no-op without buffering, but the synchronous Flush method is likewise a no-op unless flushToDisk is specified. This of course forces a subsequent call to Win32 FlushFileBuffers or Unix's fsync. Thus in the non-buffering case, the simple answer to the question is no, there is no way to ask the OS to write the bytes to the physical disk prior to closing the stream without using the synchronous Flush(bool flushToDisk) overload.

As for the more common buffering case, see FlushAsyncInternal. You'll note all this does is call the OSFileStreamStrategy.WriteAsync, i.e., it flushes its internal buffers and sends the bytes to the OS, just as if it wasn't buffering, but it doesn't go further and ask the OS to ensure they're written to disk.

The synchronous Flush(bool flushToDisk) overload in the buffering case also flushes its buffers to the OS, except synchronously. It then goes further and calls OSFileStreamStrategy.Flush(flushToDisk) - which, again, is itself a no-op unless flushToDisk is true.

So all of this is a lengthy way of saying that FlushAsync, in all cases, only guarantees that bytes have been sent to the operating system, but does not guarantee the operating system has written those bytes to disk. If you want to ask the operating system to do that, you need to call the synchronous Flush(bool flushToDisk) overload. (I say "ask" because, even then, there is never a 100% guarantee the bytes are on the hardware, especially in NAS situations).

The best approach, if you want to stay as asynchronous as possible, is therefore to write this:

await stream.FlushAsync();
await Task.Run(() => stream.Flush(true));

The two lines are actually not redundant! If you call stream.Flush(true) without await stream.FlushAsync() first, the former will synchronously send its data to the OS, blocking the thread, rather than using the more efficient asynchronous OS write functions. And if you don't use the second line, the OS will not be told to flush its own buffers to disk.

Unfortunately both FlushFileBuffers and fsync are synchronous, so you will want to wrap stream.Flush(true) in the Task.Run to avoid blocking the calling thread, but there's no way to do this in a truly asynchronous way at the hardware level (which is rather unfortunate, since at that level of course the write to disk is happening asynchronously).