I am trying to run an aggregation on a big collection using Mongo Java Driver 3.0.4. On small samples of my collection everything works great, but when I tried to execute it on the whole collection I ended up with MongoCursorNotFoundException. I found out it is a problem of the Cursor that times out and gets closed by the server.
However, I cannot understand how to set this option. The aggregate() function returns an AggregateIterable, which only has useCursor method that seems kinda relevalt. On the other hand, the find() function returns FindIterable, which has a handy noCursorTimeout(Boolean) method. I cannot see why it is so simple on find, but there's no obvious way for this option on aggregate. How should I make sure the Cursor won't fail after a while?
My code so far is this.
AggregateIterable<Document> iterable = db.getCollection("tweets").aggregate(asList( new Document("$sort", new Document("timestamp_ms", 1)),
new Document("$group", new Document("_id", "$relatedTrend")
.append("count", new Document("$sum", 1))
.append("tweets", new Document("$push", new Document("timestamp_millis", "$timestamp_ms")))))).allowDiskUse(true);
iterable.forEach(new Block<Document>() {
@Override
public void apply(final Document document) {
//parse field "tweets" of document and do a lot of calculations.
}
});
Check this MongoDB JIRA ticket. You can increase the idle cursor time out when starting an instance of mongod or mongos:
mongod --setParameter cursorTimeoutMillis=<num>
or
mongos --setParameter cursorTimeoutMillis=<num>
If this is not an option, you can also run the following shell command:
use admin
db.runCommand({setParameter:1, cursorTimeoutMillis: <num>})
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With