Go Patterns: Embedding Static Files

by about Go, Software Engineering in Technology

Go creates a single, statically linked binary. This makes it extremely easy to distribute applications. How do we add static resources, such as text files, images, and the likes, to our application without losing that advantage?

Imagine the following situation: you are writing a web service in Go that has an API, but also provides a web interface. This web interface has a bunch of HTML files, CSS, some JavaScript, etc. How do you ship this software? Do you provide the HTML and CSS files along with your binary? Do you bake the code for the frontend into your Go code? That would be a nightmare to maintain, unless there is a tool for that.

Thankfully, the community has stepped up and provided exactly that functionality in the form of countless libraries using the go:generate functionality and Go has added this feature natively in 1.16.

The basics ▲ Back to top

Let’s say you want to embed a single file. That’s fairly easy, we add a go:embed comment to our code and then we can use embed.FS type to access the embedded “filesystem”.

import "embed"

//go:embed hello-world.txt
var fs embed.FS

func main() {
    contents, err := fs.ReadFile("hello-world.txt")
    if err != nil {
        // If the file was not found, handle here.
        panic(fmt.Errorf("hello-world.txt was not found (%w)", err))
    }
    print(string(contents))
}

Adding multiple files ▲ Back to top

Adding multiple files is just as easy. You can add a wildcard, or even add multiple embed rules.

Creating a webserver ▲ Back to top

Most use cases for embedding content will be serving this content using a web server. Go has a powerful built-in HTTP server that you can use to serve these files.

We can create a filesystem suitable for the webserver library using the http.FS() function. Then we can create an HTTP handler using the http.FileServer() function. Finally, we can start a webserver using this handler.

You can also use http.StripPrefix() to remove a prefix from the embedded filesystem.

If you create the referenced files you can now run the application and the webserver should be available at http://localhost:8080

Adding a 404 handler ▲ Back to top

The default HTTP handler for the filesystems described above doesn’t provide handling for files that don’t exist. In order to do that we need to add our own handler wrapper.

We start by creating our wrapper struct:

type handlerWrapper struct {
    backend http.Handler
    fs      http.FileSystem
}

func (h *handlerWrapper) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    // Add code here in a moment
}

Once we have this wrapper let’s modify our code from before to use the wrapper as a main handler.

func main() {
    httpFS := http.FS(fs)
    handler := http.FileServer(httpFS)
    http.ListenAndServer(":8080", &handlerWrapper{
        fs: httpFS,
        backend: handler,
    })
}

Now that the wrapper is in place we need to create a dummy response writer that will capture the output from the underlying handler. We need this because we want to catch the 404 (file not found) errors from the underlying handler.

This is relatively simple, we need to create a struct with the three components we want to capture and the required receiver functions we need to handle them:

type writerWrapper struct {
    status int
    header http.Header
    body   *bytes.Buffer
}

func (w *writerWrapper) Header() http.Header {
    return w.header
}

func (w *writerWrapper) Write(bytes []byte) (int, error) {
    return w.body.Write(bytes)
}

func (w *writerWrapper) WriteHeader(statusCode int) {
    w.status = statusCode
}

Furthermore, we will need an additional helper function that writes the captured data into the real ResponseWriter. This will be used in non-404 cases.

This function consist of 3 parts: first, we write the captured HTTP status code. Then we write the captured headers. Since headers can have multiple values we need two for loops here. Finally, we write the captured body.

Now everything is ready for working on our ServeHTTP() function we created before. The first step will be simply passing through the captured content:

func (h *handlerWrapper) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    writer := &writerWrapper{
        status: 200,
        header: map[string][]string{},
        body:   &bytes.Buffer{},
    }
    h.backend.ServeHTTP(writer, r)
    _ = writer.WriteToResponse(w)
}

That’s it, if you now run the application it should work as before. However, we want to only return the captured content if the status code is not a 404. Let’s modify the code to that effect:

func (h *handlerWrapper) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    writer := &writerWrapper{
        status: 200,
        header: map[string][]string{},
        body:   &bytes.Buffer{},
    }
    h.backend.ServeHTTP(writer, r)
    if writer.status != 404 {
        _ = writer.WriteToResponse(w)
    }
}

If you now type a non-existent URL into your browser, you should get a white page. Instead of the white page we want to open 404.html from our virtual filesystem. (Don’t forget to create this file and add it to your //go:embed rules.) Once it is open, we want to read its contents and then write them to the browser.

func (h *handlerWrapper) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    writer := &writerWrapper{
        status: 200,
        header: map[string][]string{},
        body:   &bytes.Buffer{},
    }
    h.backend.ServeHTTP(writer, r)
    if writer.status != 404 {
        _ = writer.WriteToResponse(w)
    }
    errFH, err := fs.Open("404.html")
    if err != nil {
        panic(fmt.Errorf("failed to open 404.html"))
        return
    }
    defer func() {
        _ = errFH.Close()
    }()
    data, err := ioutil.ReadAll(errFH)
    if err != nil {
        _ = writer.WriteToResponse(w)
        return
    }
    w.WriteHeader(404)
    _, _ = w.Write(data)
}

That’s it! If you did everything correctly, you now have a working embedded file with proper 404 handling.