NamedTemporaryFile exists, but external program can't access it

Question

This is a follow-up of sorts to this question about using NamedTemporaryFile()

I have a function that creates and writes to a temporary file. I then want to use that file in a different function, which calls a terminal command that uses that file (the program is from the Blast+ suite, blastn).

def db_cds_to_fna(collection="genes"):  # collection gets data from mongoDB

    tmp_file = NamedTemporaryFile()
    # write stuff to file

    return tmp_file

def blast_all(blast_db, collection="genes"):        

    tmp_results = NamedTemporaryFile()    
    db_fna = db_cds_to_fna(collection) # should return another file object

    Popen(
        ['blastn',
         '-query', db_fna.name,
         '-db', blast_db,
         '-out', tmp_results.name,
         '-outfmt', '5']  # xml output
    )

    return tmp_results

When I call blast_all, I get an error from the blastn command:

Command line argument error: Argument "query". File is not accessible:  `/var/folders/mv/w3flyjvn7vnbllysvzrf9y480000gn/T/tmpAJVWoz'

But, just prior to the Popen call, if I do os.path.isfile(db_fna.name) it evaluates to True. I can also do

print Popen(['head', db_fna.name]).communicate(0)

And it properly spits out the first lines of the file. So the file exists, and it's readable. Further, I use the same strategy to call a different program from the same blast+ suite (makeblastdb, see question linked at the top) and it works. Is there possibly some problem with permissions? FWIW blastn returns the same error if the file doesn't exist, but it seems clear that I'm correctly creating the file and it's readable when I make the Popen call, so I'm stumped.

Answer 1

I think the problem may be that the OS has not synced the file to disk. After you write to the file descriptor do:

tmp_file.flush()
os.fsync(tmp_file)

https://docs.python.org/3/library/os.html#os.fsync