golang Writer.Write() from bufio package vs WriteFile() from ioutil package

Question

I'm confused when I should actually be using bufio package over the ioutil. for example when writing a file or reading a file. I have the scenario where there are multiple functions and APIs that process on the same data stage by stage. I'm not sure if choosing bufio in this case over ioutil helps? please suggest.

Answer 1

The intention of the bufio package is what it states (https://pkg.go.dev/bufio) - implements buffered I/O. So for writing, if you don't flush, data will remain in the buffer as indicated in this example. Bufio's Write also requires an object which implements the Writer interface.

Whereas ioutil doesn't have buffering etc. - you write directly to the named file, without having to open it, like:

myData := []byte("Testing\ngo\n")
err := ioutil.WriteFile("/tmp/data1", myData, 0644)

So as one usecase, if you have all of your data ready and need to write to a file - just one-time, then ioutil is a convenient choice.

However if your data gets generated as your code progresses, then bufio is a more suitable option, you can use WriteString as many times as you need and then finally call flush.

Similarly for reading, for ioutil, the Read methods read the entire data all at once, this may not be suitable for very large files, but could be desirable/acceptable in some other cases. Whereas bufio provides you methods where you have more control on how much data you want to read, it provides helpful methods for reading line by line, split by some other token etc.

Here's a program on playground which illustrates writes using both these packages.