Java recognizes some files and directories but still thinks they do not exist

Question

I'm using Java 17 on Windows 10 with NTFS. There is a file A:\foo.bar that shows up in a Java Files.list() operation, but that when I try to read it using Java, it throws a java.nio.file.FileSystemException:

A:\foo.bar: The process cannot access the file because it is being used by another process

That's fine. There is another low-level process which has locked the file. When I turn off that other process, Java can access the file just fine. In fact even Robocopy, when trying to access this file, will skip the file (actually it will appear that Robocopy has copied the file, but it hasn't). So no mystery so far—another process is locking the file for exclusive access.

But here's the strange part. For the most part the file appears normal to Java:

• As I mentioned the file A:\foo.bar shows up in a Files.list() of A:\.
• If I call Files.isRegularFile(fooBarFile), it returns true as expected.
• If I call Files.isReadable(fooBarFile), it turns false as I might expect (and which is useful in this case).
• If I call Files.readAttributes(fooBarFile, "*") I see the attributes (timestamps, etc.).
• If I call Files.readAttributes(fooBarFile, DosFileAttributes.class) it returns the DOS attributes.

But if I call Files.exists(fooBarFile) it returns false! So it would appear that a file that is locked for exclusive access by another process will return false for exists(), which to me doesn't seem to follow the semantics of the exists() method as explained in its API.

As it is, it does seem useful to see if a file is not accessible by checking to see if exists() returns false yet isRegularFile() returns true; nevertheless that was unexpected and seems to be undocumented. Is this expected behavior? Is it documented? Does it work the same on other platforms, e.g. Linux?

Finally I note that Files.notExists(fooBarFile) returns false as well, so Java is not saying that the file is nonexistent, merely that it does not exist. Hmmm … the only way I can make that make sense is if exists() means "accessible", but the API contract for exists() does not talk about accessibility. The notExists() documentation adds:

Note that this method is not the complement of the exists method. Where it is not possible to determine if a file exists or not then both methods return false.

That doesn't seem to apply to this situation either. So although this behavior is useful, it is unexpected, doesn't seem to be documented, and therefore I'm hesitant about relying too much on it. Can anyone provide more information or better yet authoritative documentation?

Update: A similar thing seems to happen with unreadable directories. For example the A:\System Volume Information directory is marked as "hidden" and "read-only" on Windows. It similarly throws a java.nio.file.FileSystemException exception if you try to access it. But Files.exists() returns false, even though Files.isDirectory() returns true! In fact it behaves exactly as the bullet points above, except that isDirectory() returns true instead of isRegularFile().

Thus it seems like Files.exists() is duplicating Files.isReadable() (at least on OpenJDK 17 on Windows), even though that behavior doesn't seem to follow the Files.exists() API contract. Is this a JDK bug?

Answer 1

Thus it seems like Files.exists() is duplicating Files.isReadable() (at least on OpenJDK 17 on Windows), even though that behavior doesn't seem to follow the Files.exists() API contract. Is this a JDK bug?

TL;DR

This is the way it was designed to behave and it follows the API contract. One major reason for why we have 2 methods, is that both methods return true if they are sure about the question.

• Files.exist() true means file definitely exists.
• Files.notExists() true means file definitely not exists.

But false does not mean otherwise. It could as well mean that the system can't determine the outcome and it returns false as the safest return if it can't throw such a checked exception.

• If you want to do some action which is based on the condition that the file does not exist in the system, you should never use If (!Files.exists(file)) {...}. In this case let's say you want to create a new file and you want to make sure that another file does not already exist you should always check with If(Files.notExists(file)){...}

• On the other side if you want to do some action which is based on the condition that the file exist in the system, you should never use If (!Files.notExists(file)) {...}. In this case let's say you want to read/modify a file and you want to make sure that the file already exist you should always check with If(Files.exists(file)){...}

Analytical explanation

I believe the java API doc of Files.exists(fooBarFile) could be a bit better, although the java API doc for notExists is a bit clearer for this in the following:

Note that this method is not the complement of the exists method. Where it is not possible to determine if a file exists or not then both methods return false.

Here is what I suspect is going on.

According to the method description

//  @return  
// {@code true} if the file exists; 
// {@code false} if the file does not exist or its existence cannot be determined.
public static boolean exists(Path path, LinkOption... options) {
        if (options.length == 0) {
            FileSystemProvider provider = provider(path);
            if (provider instanceof AbstractFileSystemProvider)
                return ((AbstractFileSystemProvider)provider).exists(path);
        }

        try {
            if (followLinks(options)) {
                provider(path).checkAccess(path); <-------------- if no options provided this is executed
            } else {
                // attempt to read attributes without following links
                readAttributes(path, BasicFileAttributes.class,
                               LinkOption.NOFOLLOW_LINKS);
            }
            // file exists
            return true;
        } catch (IOException x) {
            // does not exist or unable to determine if file exists
            return false;  <----- AccessDeniedException is causing return false
        }

    }

If someone inspects the code he will see the meaning of @return {@code false}... or its existence cannot be determined.

could be further explained from the line of .checkAccess(path) which mentions in the API doc:

 * @throws  AccessDeniedException
 *          the requested access would be denied or the access cannot be
 *          determined because the Java virtual machine has insufficient
 *          privileges or other reasons. <i>(optional specific exception)</i>

So in case a AccessDeniedException is thrown which is also a subclass of FileSystemException (which also the questioner is getting from another command during file read), then Files.exists(fooBarFile) is supposed to return false because of that exception which does not allow the JVM to determine if the file definitely exists.

Question also mentions:

Files.notExists(fooBarFile) returns false as well

which I think originates also from the same point.

public static boolean notExists(Path path, LinkOption... options) {
        try {
            if (followLinks(options)) {
                provider(path).checkAccess(path);  <---------- If no options provided this line executes
            } else {
                // attempt to read attributes without following links
                readAttributes(path, BasicFileAttributes.class,
                               LinkOption.NOFOLLOW_LINKS);
            }
            // file exists
            return false;
        } catch (NoSuchFileException x) {
            // file confirmed not to exist
            return true;
        } catch (IOException x) { 
            return false;   <------ In case of AccessDeniedException or some other IOException the return is false.
        }
    }

Which verifies that both methods exists and notExists are designed to return false in case AccessDeniedException or some other IOException is thrown, as this would be translated into , JVM can't determine if the file exists or not.

So each method gives the safest return it could give in this uncertain condition, considering it can't throw such an exception. So in case

• it is exists() it returns false if it can't determine, so that the developer would not be sure that the file exists and move forward with actions such as reading the file.
• it is notExists() it returns false if it can't determine, so that developer would not be sure that the file does not exist and move forward with actions such as creating a new file.

So both methods shall be used appropriately and trusted for a definite answer only if they return true. false does not mean negative answer to exists or notExists. And someone should never be based on the outcome of !Files.exists(file) and !Files.notExists(file), at least as a logical inversion of the answer of the method.