Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to fsync a directory?

I have a write-ahead log that stores some metadata using atomic renames. Currently I am only fsyncing the file before rename, but it is my understanding that I should also be fsyncing the directory after rename to ensure this atomic operation is flushed to disk, before reporting back to the client that the operation succeeded.

I have the following:

import fs from "fs/promises";

I tried two ways to fsync a directory.

  1. Calling sync on the fs.Dir object:

    const dir = await fs.opendir(...);
    await dir.sync(); // error: no sync method exists
    
  2. Calling fsync and passing in file descriptor:

    const dir = await fs.opendir(...);
    await fs.fsync(dir.fd); // error: no fsync method exists
    

How do I fsync my directory using Node.js fs/promises or is there any other viable solution?

This Reddit post is related but isn't referring to the promises API which is what I am working with.


You might ask, why do I believe I need to fsync a directory?

According to the linux man pages:

As well as flushing the file data, fsync() also flushes the metadata information associated with the file (see inode(7)).

Calling fsync() does not necessarily ensure that the entry in the directory containing the file has also reached disk. For that an explicit fsync() on a file descriptor for the directory is needed.

like image 763
David Callanan Avatar asked Dec 21 '25 22:12

David Callanan


1 Answers

Attempting to sync/fsync a directory is a flawed concept because these methods are used to flush IO buffers and directories are essentially 0 bytes. All information pertaining to a directory is stored within the filesystem which is inherently atomic. Specifically, in this context, renaming a directory is atomic. Somehow calling fsync on that directory after the rename would not accomplish anything beneficial. At best it expedites the operation, which would have an effect in catastrophic scenarios.

fs/promises of course allows you to call fsync on normal files; this would be the FileHandle#sync method. To obtain a FileHandle, you can use open.

const file = await fs.open(...);
await file.sync();

There is nothing similar to the code you posted inside fs or fs/promises. The opendir method (not openDir) is analogous to opendir in glibc and does not provide a file descriptor. It is used to enumerate the contents of a directory.

Of course, in C/C++, you can open a directory to yield an int file descriptor. NodeJS is providing a higher-level abstraction that plays to the language's strengths.

like image 128
Wasabi Thumbs Avatar answered Dec 24 '25 10:12

Wasabi Thumbs