Java Camel RouteBuilder picks up file before copying is complete

Question

I am using Ubuntu (in case it will make a difference) and I am trying use Camel to send files to processor from one folder. But the problem is that when I am saving this file in the folder (takes about 5-10 seconds) Camel picks it up straight away. To simulate the process I am using gedit with txt file with ~500k rows so it will take some time to save.

I have tried adding options:

from("file:src/Data/new/?readLock=changed&readLockMinAge=3m")

I have tried using .filter(header("CamelFileLastModified").isGreaterThan(new Date(System.currentTimeMillis()-120000))) to give 2 minute delay.

Nothing seems to influence its behaviour, it picks it up straight away, throws an exception because of some checks while processing file and moves it to the Error folder.

I know there is an issue with FTP file transfers which I will have to face later on, but I can not even get it working on local file system.

Any help will be appreciated!

SOLVED

from("file:src/Data/new/?readLock=changed&readLockMinAge=3m")

Parameters actually work as they should. I was using Jetty to run the project and I should have done whole project clean/install after any amendments.

I had to amend parameters a bit to:

from("file:src/Data/new/?readLock=changed&readLockTimeout=65000&readLockMinAge=1m")

because it was complaining that readLockTimeout should be more than readLockCheckInterval + readLockMinAge.

Answer 1

Have a look into the documentation:

Avoid reading files currently being written by another application Beware the JDK File IO API is a bit limited in detecting whether another application is currently writing/copying a file. And the implementation can be different depending on OS platform as well. This could lead to that Camel thinks the file is not locked by another process and start consuming it. Therefore you have to do you own investigation what suites your environment. To help with this Camel provides different readLock options and doneFileName option that you can use. See also the section Consuming files from folders where others drop files directly.

So I think the doneFileName option will solve your problem.