When do I call boost::asio::streambuf::consume() and boost::asio::streambuf::commit()?

Question

I'm trying to understand the boost::asio::streambuf::consume() and boost::asio::streambuf::commit() calls. In the docs, we have the examples,

boost::asio::streambuf b;
std::ostream os(&b);
os << "Hello, World!\n";

// try sending some data in input sequence
size_t n = sock.send(b.data());

b.consume(n); // sent data is removed from input sequence

and

boost::asio::streambuf b;

// reserve 512 bytes in output sequence
boost::asio::streambuf::mutable_buffers_type bufs = b.prepare(512);

size_t n = sock.receive(bufs);

// received data is "committed" from output sequence to input sequence
b.commit(n);

std::istream is(&b);
std::string s;
is >> s;

I understand these two calls as much as I understand what the documentation says about them - call consume() to remove characters from the input sequence inside the boost::asio::streambuf, and call commit() to move characters from boost::asio::streambuf's output sequence to its input sequence. Fair enough.

When do I actually call these? Looking at the boost::asio::read_until() source, we have

template <typename SyncReadStream, typename Allocator>
std::size_t read_until(SyncReadStream& s,
    boost::asio::basic_streambuf<Allocator>& b, char delim,
    boost::system::error_code& ec)
{
  std::size_t search_position = 0;
  for (;;)
  {
    // Determine the range of the data to be searched.
    typedef typename boost::asio::basic_streambuf<
      Allocator>::const_buffers_type const_buffers_type;
    typedef boost::asio::buffers_iterator<const_buffers_type> iterator;
    const_buffers_type buffers = b.data();
    iterator begin = iterator::begin(buffers);
    iterator start_pos = begin + search_position;
    iterator end = iterator::end(buffers);

    // Look for a match.
    iterator iter = std::find(start_pos, end, delim);
    if (iter != end)
    {
      // Found a match. We're done.
      ec = boost::system::error_code();
      return iter - begin + 1;
    }
    else
    {
      // No match. Next search can start with the new data.
      search_position = end - begin;
    }

    // Check if buffer is full.
    if (b.size() == b.max_size())
    {
      ec = error::not_found;
      return 0;
    }

    // Need more data.
    std::size_t bytes_to_read = read_size_helper(b, 65536);
    b.commit(s.read_some(b.prepare(bytes_to_read), ec));
    if (ec)
      return 0;
  }
}

You can see that, as the documentation says, boost::asio::read_until() is implemented in terms of SyncReadStream's read_some().

To me, that says that

1. SyncReadStream::read_some() does NOT call boost::asio::streambuf::commit()
2. boost::asio::read_until() DOES call boost::asio::streambuf::commit()
3. Neither of these seem to be documented - neither in boost::asio::read_until()'s documentation, nor in SyncReadStream's docs.
4. I don't know whether I'm supposed to be calling boost::asio::streambuf::commit() at all?

With my synchronous code I certainly don't seem to need it, not when I'm calling the free functions boost::asio::read() and boost::asio::read_until(). I have it in my async code in my handlers mostly because the examples I used had it, but I'm not sure about calling it then, either. When I try to use a boost::asio::streambuf with stringstream's and std::string's, commit() doesn't seem to play a role - nothing gets halted or stuck without calling commit() on the streambuf.

Can anyone sort this out for me?

Answer 1

Asio defines a number of auxiliary free functions (read_xxx) that accept asio::streambuf, and they care to prepare and commit it.

On the other hand, if you'd like to use asio::streambuf with lower-level functions that accept a model of MutableBufferSequence concept, you have to call streambuf::prepare(), which returns an objects that meets MutableBufferSequence concept, pass this object as a buffer, and after the function fills it - call commit().

In both cases, after you've read n bytes of data from the streambuf, you have to call consume(n) - in order to consume the input sequence.