Read the large document using marklogic Nodejs API

Question

I am trying to read xml/json documents using Node.js from MarkLogic. I have downloaded Node.js API code base from the following URL.

https://github.com/marklogic/node-client-api

It's working fine for small documents like 500KB. But our requirement is to read large documents, like 2 MB or 10 MB.

Regarding this, we have two cases, as mentioned below:-

Case 1:- Whenever I am trying to read a document using MarkLogic Node.js API,I should get more than one chunk, but I got only one chunk as response. So for this reason it is not working.

var chunks = 0;
var length = 0;
db.documents.read("test.xml").stream('chunked').
  on('data', function(chunk) {
    console.log(chunk);
    console.log(chunk.length);
    chunks++;
    length += chunk.length;
  }).
  on('error', function(error) {
    console.log(JSON.stringify(error));
  }).
  on('end', function() {
    console.log('read '+chunks+' chunks of '+length+' length');
    console.log('done');
  });

Case 2:- Whenever I am trying to read some large document( like 2 MB or 10 MB) using "http-digest-client" package then it is working fine and I get complete xml as response.

var digest = require('http-digest-client')('<Username>', '<password>');
digest.request({
  host: '<Host server>',
  path: '/v1/documents?uri=test.xml',
  port: 8007,
  method: 'GET',
}, function (res) {
  reply(res);
});

I have tested this with large document in the below mentioned example (please refer below url), but I get the same response which I have described in case 1 above.

https://github.com/marklogic/node-client-api/blob/develop/examples/read-stream.js#L27

As per my requirement, I would like to read the large document using MarkLogic Node.js API (Case 1).

1. How can I read a large document using MarkLogic Node.js API?
2. Is there any option to increase pool memory size or any other memory size?
3. Is this issue related memory size?

Answer 1

Thanks for the code example. I'm able to reproduce the problem.

Please file an issue against the MarkLogic Node.js API at:

https://github.com/marklogic/node-client-api/issues