Moving data around is one of the most common things you do when programming. That’s why no matter the language you use, you must master handling the data flow.
I’m starting a small series about working with I/O in Go today.
This first part aims to get you in the right mindset and provide you with a minimal toolset big enough to make you dangerous.
Everything is a bunch of bytes Link to heading
Someone was complaining on LinkedIn that it’s confusing that len gives the number of bytes, not the number of characters of a string:
package main
import "fmt"
func main() {
str := "ana"
fmt.Println(len(str)) // 3
str = "世界"
fmt.Println(len(str)) // 6 not 2
}
This may be confusing to someone because of the wrong mental model. In computing, every piece of data is just a series of bytes. If you want its length, then you need the number of bytes, not the number of interpreted things in that data.
If I load the below image in Go and run len on the data, I will not get 1 because there’s 1 cat in the picture, but 293931 because the image has 293931 bytes.
Remember: every piece of data you are manipulating, whether that’s a string, an image, or a file is just a bunch of bytes.
Readers and Writers Link to heading
Data movement comprises two fundamental parts: reading and writing. You read from somewhere, apply some transformations or whatnot, and later on, you write that final data somewhere.
Because reading and writing are fundamental operations, Go offers us two abstractions: io.Reader and io.Writer. From here on, for the sake of readability, I will refer to them as Reader and Writer.
Remember: a Reader is something you can read from, and a Writer is something you can write to.
I emphasized that because when I started with Go, this naming confused me. When I hear about a writer, I think of someone who writes. But in Go, a Writer is something you can write to.
When defining interfaces in Go, you don’t define what something is but what it provides—behavior, not things! There was my misunderstanding.
For example, we don’t have a File interface with the Read and Write methods, but we have Reader and Writer interfaces that are implemented by a concrete file type.
When working with I/O in Go, you don’t care what something is, but whether you can read from it or write to it. As long as it implements the Reader interface, you can read from it, and if it implements the Writer interface, you can write to it.
If you have some bytes, you can write them to any Writer. It doesn’t matter if the bytes represent a file, a network connection, an HTTP response, or something else.
The minimal I/O toolkit Link to heading
Now let’s add the first types and functions to our toolset as promised.
bytes.Buffer Link to heading
As I said above, every piece of data is just a bunch of bytes. Usually, that’s represented as a []byte, but the slice
of
bytes does not implement Reader and Writer interfaces.
One type I often use is the bytes.Buffer because it’s like a []byte that implements
both Reader and Writer interfaces
simplifying reading and writing to it.
io.Copy Link to heading
Do you have a Reader and want to copy its contents to a Writer? io.Copy does the trick!
Bellow we create a buffer, add a string in it, and then copy that data to os.Stdout which is just a file in the end:
package main
import (
"bytes"
"io"
"os"
)
func main() {
var buf bytes.Buffer
buf.WriteString("hello world\n")
io.Copy(os.Stdout, &buf)
}
fmt.Fprint Link to heading
Sometimes you just want to print some strings to a Writer. For this, you can use fmt.Fprint% functions:
package main
import (
"bytes"
"fmt"
"os"
)
func main() {
var buf bytes.Buffer
// Fprintf can be called to print a string to any io.Writer
fmt.Fprintf(&buf, "hello world!") // on bytes.Buffer
fmt.Println(buf.String())
fmt.Fprintf(os.Stdout, "hello world!\n") // on a *os.File
}
I like these functions also when I’m writing webservers. A /health endpoint can be as simple as this:
package handlers
import (
"fmt"
"net/http"
)
func HealthHandler() http.Handler {
return http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "OK") // write OK string to response body
},
)
}
And because you can write straight to the response body, it’s also handy for print debugging in your browser.
io.ReadAll Link to heading
This method is useful when you want to read all bytes from a certain Reader:
package main
import (
"fmt"
"io"
"strings"
)
func main() {
r := strings.NewReader("Hello World!")
data, _ := io.ReadAll(r)
fmt.Println(data) // [72 101 108 108 111 32 87 111 114 108 100 33]
fmt.Println(string(data)) // Hello World!
}
If you’re dealing with a huge file, it might not always be a good idea to read the full content at once. We will see in future posts what alternatives we have. However, io.ReadAll is a very handy function to have in your toolkit.
Conclusion Link to heading
To recap, here’s the most important information covered in this article:
At the fundamental level, every piece of data is just a bunch of bytes.
Most of what we do when programming is moving data around.
Moving data is composed of two parts: reading and writing. Go offers us io.Reader and io.Writer abstractions for these operations.
A Reader is something you can read from, and a Writer is something you can write to.
As long as something implements io.Reader you can read from it and if it implements io.Writer you can write to it.
We also added a bunch of useful types and functions to our toolkit:
Now you’re equipped with the basic utilities and information to deal with I/O in Go. You still have much to learn, but don’t underestimate the power these few tools will give you. ‘Till next time!