How to push data into Java stream regularly? [closed]

Question

Normally one should get data from a database using a stream and then send it to Apache POI using a stream, etc. I mean, data should be in stream format from start to end. However my database codes are not written as stream and they have complicated sources from various different types from databases. Thus I need to query my data source like this getData(int page, int perPage). And then I want to forward the results to the stream. Like this:

for(int i = 0; i < 5000; i++) {
    stream.add(getData(i, 10000));
}

So my question is how can I push data into the stream on the fly without using too much RAM?

Answer 1

You can just implement an iterator and wrap it in a stream right?

Stream<T> stream = stream(new Iterator<T>() {
    private Iterator<T> currentBatch;
    private int page;

    @Override
    public boolean hasNext() {
        if (currentBatch != null && currentBatch.hasNext()) return true;
        currentBatch = getData(page, BATCH_SIZE).iterator();
        return currentBatch.hasNext();
    }

    @Override
    public T next() {
        if (!hasNext()) throw new NoSuchElementException();
        return currentBatch.next();
    }
});

private static <T> Stream<T> stream(Iterator<T> iterator) {
    return StreamSupport.stream(Spliterators.spliteratorUnknownSize(iterator, UNKNOWN_CHARACTERISTICS), false);
}