remote neo4j data base: import large amount of data

Question

I need to import about one million of nodes and rougly twice as many relationships between them to a remote neo4j database. The thing that matters to me is: performance

I found the java-rest-binding which seems to offer just what I need. However:

• it's too slow (about 1000 nodes / second, even if the remote database is in fact on my local machine)
• it's buggy, i.e. it just hangs on committing the batch operation if I try to add more than ~900 nodes at once

How could I achieve the speed I need? And, why the java-rest-binding not working / slow?

(I would like to avoid having to deal with HTTP requests and such myself, as it is suggested here in the docs)

I'm happy to post code or more details. just tell me!

Thanks a lot for your help!

Nandaloo

Answer 1

I've done some insertion tests as well, and the max I got (without going a lot into fine optimizations) was something like 500 nodes with 10 properties + 500 relationships / s

I am not sure you can go beyond what you already got (1000 nodes / s) using the REST interface. You could probably try using a custom extension that would probably go faster than the REST API, provided that the data to load is local if you cannot shutdown your DB and use the BatchInserter (which is very very fast btw).