Turning a callback which gets called multiple times into a generator function

Question

I have a function that fetches rows of a database asynchronously, and calls a callback function for each row. I am trying to write a wrapper which is a generator function which yields each time a row is returned, but I am not seeing how to properly yield.

The original code looks something like this:

db.each(query, (err, row) => {
  // do something here with row
}, () => {
  // called after the last row is returned
})

I'm familiar with how a generator works, but the yield seems to belong in the generator function itself, not in an anonymous function. So I think something like this wouldn't work:

function* dbEach(db, query) {
    db.each(query, (err, row) => {
      yield row
    })
}

When I actually try this I get an error "Unexpected identifier".

I looked a bit further and it appears that ES2018 now has asynchronous iterators which are supposed to make this possible. However, I'm having trouble wrapping my head around how exactly I can use async iterators in the case where I already have a callback which is getting called multiple times.

Answer 1

You can make the generator async, then await a Promise that resolves with all rows (so that you have reference to a rows variable on the top level of dbEach), and then you can yield each row in that rows array:

async function* dbEach(db, query) {
  const rows = await new Promise((resolve, reject) => {
    const rows = [];
    db.each(query, (err, row) => {
      if (err) reject(err);
      else rows.push(row);
    }, () => resolve(rows));
  });
  for (const row of rows) {
    yield row;
  }
}

Use with:

for await (const row of dbEach(...)) {
  // do something
}

It looks like .each is designed for running a callback on each row, which isn't exactly optimal for what you're trying to accomplish here with a generator - if possible, it would be great if there was a method in your database that allows you to get an array of rows instead, for example:

async function* dbEach(db, query) {
  const rows = await new Promise((resolve, reject) => {
    db.getAllRows(query, (err, rows) => {
      if (err) reject(err);
      else resolve(rows);
    });
  });
  for (const row of rows) {
    yield row;
  }
}

Though, I don't think that a generator helps a lot here - you may as well just await a Promise that resolves to the rows, and iterate over the rows synchronously:

function getRows(db, query) {
  return new Promise((resolve, reject) => {
    db.getAllRows(query, (err, rows) => {
      if (err) reject(err);
      else resolve(rows);
    });
  });
}

const rows = await getRows(...);
for (const row of rows) {
  // ...
}