Using the common I/O interfaces

Go provides a number of I/O interfaces used throughout the standard library. It is a best practice to make use of these interfaces wherever possible rather than passing structs or other types directly. Two powerful interfaces we explore in this recipe are the io.Reader and io.Writer interfaces. These interfaces are used throughout the standard library and understanding how to use them will make you a better Go developer.

The Reader and Writer interfaces look like this:

type Reader interface {
        Read(p []byte) (n int, err error)
}

type Writer interface {
        Write(p []byte) (n int, err error)
}

Go also makes it easy to combine interfaces. For example, take a look at the following code:

type Seeker interface {
        Seek(offset int64, whence int) (int64, error)
}

type ReadSeeker interface {
        Reader
        Seeker
}

The recipe will also explore an io function called Pipe():

func Pipe() (*PipeReader, *PipeWriter)

The remainder of this book will make use of these interfaces.

Getting ready

Configure your environment according to these steps:

1. Download and install Go on your operating system at https://golang.org/doc/install and configure your GOPATHenvironment variable.

2. Open a terminal/console application, navigate to your GOPATH/src directory, and create a project directory such as $GOPATH/src/github.com/yourusername/customrepo.

All code will be run and modified from this directory.

3. Optionally, install the latest tested version of the code using the following command:

      go get github.com/agtorre/go-cookbook/

How to do it...

These steps cover writing and running your application:

1. From your terminal/console application, create a new directory called chapter1/interfaces.
2. Navigate to that directory.

Copy tests from https://github.com/agtorre/go-cookbook/tree/master/chapter1/interfaces, or use this as an exercise to write some of your own code.

1. Create a file called interfaces.go with the following contents:

        package interfaces

        import (
                "fmt"
                "io"
                "os"
        )

        // Copy copies data from in to out first directly,
        // then using a buffer. It also writes to stdout
        func Copy(in io.ReadSeeker, out io.Writer) error {
                // we write to out, but also Stdout
                w := io.MultiWriter(out, os.Stdout)

                // a standard copy, this can be dangerous if there's a 
                // lot of data in in
                if _, err := io.Copy(w, in); err != nil {
                    return err
                }

                in.Seek(0, 0)

                // buffered write using 64 byte chunks
                buf := make([]byte, 64)
                if _, err := io.CopyBuffer(w, in, buf); err != nil {
                    return err
                }

                // lets print a new line
                fmt.Println()

                return nil
        }

2. Create a file called pipes.go with the following contents:

        package interfaces

        import (
                "io"
                "os"
        )

        // PipeExample helps give some more examples of using io  
        //interfaces
        func PipeExample() error {
                // the pipe reader and pipe writer implement
                // io.Reader and io.Writer
                r, w := io.Pipe()

                // this needs to be run in a separate go routine
                // as it will block waiting for the reader
                // close at the end for cleanup
                go func() {
                    // for now we'll write something basic,
                    // this could also be used to encode json
                    // base64 encode, etc.
                    w.Write([]byte("testn"))
                    w.Close()
                }()

                if _, err := io.Copy(os.Stdout, r); err != nil {
                    return err
                }
                return nil
        }

3. Create a new directory named example.
4. Navigate to example.

5. Create a main.go file with the following contents and ensure that you modify the interfaces imported to use the path you set up in step 2:

        package main

        import (
             "bytes"
             "fmt"

             "github.com/agtorre/go-cookbook/chapter1/interfaces"
        )

        func main() {
                in := bytes.NewReader([]byte("example"))
                out := &bytes.Buffer{}
                fmt.Print("stdout on Copy = ")
                if err := interfaces.Copy(in, out); err != nil {
                        panic(err)
                }

                fmt.Println("out bytes buffer =", out.String())

                fmt.Print("stdout on PipeExample = ")
                if err := interfaces.PipeExample(); err != nil {
                        panic(err)
                }
        }

6. Run go run main.go.
7. You may also run these:

      go build 
      ./example

You should see the following output:

        $ go run main.go
        stdout on Copy = exampleexample
        out bytes buffer = exampleexample
        stdout on PipeExample = test

8. If you copied or wrote your own tests, go up one directory and run go test, and ensure all tests pass.

How it works...

The Copy() function copies between interfaces and treats them like streams. Thinking of data as streams has a lot of practical uses, especially when working with network traffic or filesystems. The Copy() function also creates a multi-writer that combines two writer streams and writes to them twice using ReadSeeker. If a Reader interface were used instead rather than seeing exampleexample, you would only see example despite copying to the MultiWriter interface twice. There's also an example of a buffered write that you might use if your stream is not fit into the memory.

The PipeReader and PipeWriter structs implement io.Reader and io.Writer interfaces. They're connected, creating an in-memory pipe. The primary purpose of a pipe is to read from a stream while simultaneously writing from the same stream to a different source. In essence, it combines the two streams into a pipe.

Go interfaces are a clean abstraction to wrap data that performs common operations. This is made apparent when doing I/O operations, and so the io package is a great resource for learning about interface composition. The pipe package is often underused but provides great flexibility with thread-safety when linking input and output streams.