File create time doesn't change even after it is deleted

Question

I am using the following code:

from datetime import datetime
import time, pandas as pd, os, pickle
df = pd.DataFrame(np.arange(1,200))
fn = r'C:\z1.p'
pickle.dump(df, open(fn, 'wb'))
print(datetime.fromtimestamp(os.stat(fn).st_ctime)) 
os.remove(fn)
time.sleep(5)
pickle.dump(df, open(fn, 'wb'))
print(datetime.fromtimestamp(os.stat(fn).st_ctime))

But I get the same create time from both print statements as:

2022-03-16 08:43:30.885011
2022-03-16 08:43:30.885011

How do I make sure that new time gets printed for second print statement?

Answer 1

This is a Windows feature, called "file system tunnelling".

The apocryphal history of file system tunnelling

One of the file system features you may find yourself surprised by is tunneling, wherein the creation timestamp and short/long names of a file are taken from a file that existed in the directory previously. In other words, if you delete some file “File with long name.txt” and then create a new file with the same name, that new file will have the same short name and the same creation time as the original file. You can read this KB article for details on what operations are sensitive to tunnelling.

Why does tunneling exist at all?

When you use a program to edit an existing file, then save it, you expect the original creation timestamp to be preserved, since you’re editing a file, not creating a new one. But internally, many programs save a file by performing a combination of save, delete, and rename operations (such as the ones listed in the linked article), and without tunneling, the creation time of the file would seem to change even though from the end user’s point of view, no file got created.

...

See this archived copy of Windows NT Contains File System Tunneling Capabilities:

When a name is removed from a directory (rename or delete), its short/long name pair and creation time are saved in a cache, keyed by the name that was removed. When a name is added to a directory (rename or create), the cache is searched to see if there is information to restore. The cache is effective per instance of a directory. If a directory is deleted, the cache for it is removed.

These paired operations can cause tunneling on "name."

• delete(name)/create(name)
• delete(name)/rename(source, name)
• rename(name, newname)/create(name)
• rename(name, newname)/rename(source, name)

The idea is to mimic the behavior MS-DOS programs expect when they use the safe save method. They copy the modified data to a temporary file, delete the original and rename the temporary to the original. This should seem to be the original file when complete. Windows performs tunneling on both FAT and NTFS file systems to ensure long/short file names are retained when 16-bit applications perform this safe save operation.

One Windows function related to file tunneling is FltGetTunneledName():

The FltGetTunneledName routine retrieves the tunneled name for a file, given the normalized name returned for the file by a previous call to FltGetFileNameInformation, FltGetFileNameInformationUnsafe, or FltGetDestinationFileNameInformation.

...

To disable tunnelling:

1. Open regedit

2. Navigate here:

   HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem

3. On the Edit menu, point to New and then click DWORD Value

4. Type MaximumTunnelEntries and then press Enter

5. On the Edit menu, click Modify

6. Type 0 and then click OK

7. Restart your computer

8. Done