20,000 lines under the Go-cean

by about Go, Software Engineering in Technology

Last year I wrote two blog posts about the programming language Go. This is the follow-up after having written over 20,000 lines of Go code.

I started learning Go the programming language early last year. While I had over a decade of experience in other languages, such as Python, PHP, Java, etc. I found it extremely difficult to wrap my head around Go.

What language was it even: Procedural? Functional? Surely not, OOP, right? Right? As it turned out, I couldn’t be more wrong.

As a learning project I chose something I already implemented in Java: an SSH server that launches containers. It took me a couple of months to go from a first prototype to something you could consider close to production and I learned a lot: Go could, indeed, be used as an effective tool while adhering to the clean code principles and OOP was also possible.

As I went through the learning process I wrote two articles:

Since then I spent a few more months working on my Go skills and built what became ContainerSSH. In November last year, I got a job at Red Hat as a senior software engineer working mainly with Go. Next to my day job I also launched a major refactor of ContainerSSH for version 0.4 which took me down a rabbit hole of writing more than 20,000 lines of code in 30+ libraries over the course of 4 months.

So, what is my opinion of Go after all this? Let’s break it down.

Code organization ▲ Back to top

As you might imagine, keeping a semblance of order in a codebase of this size is tricky in any language. In Java, for example, you organize things into classes as more or less self-contained units.

In Go, however, the unit of organization is a package. With Go modules, packages are structured in such a way that any code living a folder has full access to other files in the same folder.

What’s the problem with this, you ask? If you typically work in smaller teams, or smaller codebases, this might not seem like a big deal. However, as you scale things up you can’t trust everyone with access to the code to know how the code is structured and what you are or aren’t supposed to do. Indeed, most of my work involves working on code I have never seen before.

This then leads to lengthy discussions in pull requests about code quality. The reviewers can be seen as nit-picky for complaining about seemingly unimportant details, while a contributor can come off as less skilled because they don’t know how they are supposed to write code.

This communication problem can be mitigated by a practice called defensive programming. Among other techniques, defensive programming uses access control to prevent a contributor from doing the wrong things. Think of things like the private keyword in Java.

This can be extremely difficult to achieve in Go as everything within a package (thus, a folder in most cases) can see everything. There is simply no way to create a function or data structure that is not callable within a package.

For example, the Docker library in ContainerSSH contains two layers: the upper layer is responsible for managing higher level functions, while the lower layer is responsible for talking directly with the Docker API. What prevents a contributor from sending in code that calls the Docker API from a higher level code piece? Mostly, nothing. We could turn to functional programming and create only pure functions, but I found that having no internal state is completely impractical when needing to work with external APIs.

Why didn’t I move the lower level API to a separate package then? The reason is very simple: if it’s in a separate package it’s accessible to everyone, not just the higher level layer. It effectively becomes part of the public API for the Docker library.

I’m between a rock and a hard place: neither of these solutions is good. I truly wish there was a way to restrict visibility within a package.

Object oriented programming ▲ Back to top

One of my initial mistakes was thinking that you can’t write OOP code in Go. Nothing could be further from the truth.

Object oriented programming is a pragmatic approach to solving the problem of managing state in a program. In other words, keeping track of what’s going on where. At its core objects in OOP have three parts:

  1. A data structure with specified fields and field types.
  2. A set of functions that work in the context of the aforementioned data structure, usually referred to as this.
  3. A function that takes a set of parameters and creates a copy of the data structure. This is usually referred to as a constructor, but in Go this is often referred to as a factory.

Go has all these parts. Let’s compare a Java and a Go solution to creating a simple web server:


class WebServer {
  private String listen;

  public WebServer(listen String) {
    this.listen = listen
  public void run() {
    // ...


type WebServer struct {
  listen string

func NewWebServer(listen string) *WebServer {
  return &WebServer {
    listen: listen.

func (w *WebServer) Run() {

See? Not that much of a difference. However, unlike in Java, in Go you can create a copy (instance) of the WebServer struct without using the NewWebServer() constructor since it is written with a capital letter.

Visibility in Go
Types and variables written with a capital first letter are exported and can be called / set / read from other packages.

This can lead to problems, since in the example above someone could create a copy of the struct, but the caller can’t initialize the listen field since it’s not public. Even worse, I’ve seen quite a few instances of some fields being exported, while others are not. This can lead to quite a bit of confusion around usage.

That’s not the death of defensive OOP programming, of course. The method I opted to follow is creating an interface for each class and have the factory declare the interface as a return type instead of the actual underlying struct. This makes it possible to create a clean cut, but it comes with a metric ton of boilerplate code.

Isn’t this overkill? ▲ Back to top

You may be wondering: isn’t this overkill? Surely, developers would apply common sense?

Over the past few months I had the dubious pleasure of diving deep into various projects in the Kubernetes ecosystem and I beg to differ. All too often someone just wants to fix their problem, submits a PR that technically works, but doesn’t exactly improve code quality. When layers upon layers of these less than ideal PRs build on top of each other even the maintainers of the project often give up on trying to maintain order.

I have also seen quite a few maintainers switch to ultra nit-pick mode and exacerbate the living hell out of contributors by doing rounds after rounds of reviews on how the code should look like, only to abandon the PR unmerged when they find nothing more to criticise.

In my experience, defensive programming makes it harder to contribute because it raises the learning curve a potential contributor has to go through to get their fix in. On the other hand, it makes PRs easier to merge since the structure of the code ensures a base level of quality.

Error handling ▲ Back to top

Another often-leveraged criticism of Go is its error handling. Go doesn’t have exceptions, instead it has a type that must be returned through the levels of the application. Let’s take a look at a function that performs API calls to a remote HTTP server:

func Call(
    url string,
) (Result, error) {
    if url == "" {
        return nil, fmt.Errorf("url cannot be empty")     

Seems simple enough, right? Except, a lot of Go developers do this in their code:

func GetPriceList(
    url string
) (PriceList, error) {
    result, err := Call(url, ...)
    if err != nil {
        return nil, err

Finally, somewhere on the top level there will be a logging function that writes the received error to the standard error or a log file. Let’s look at the log line:

url cannot be empty

Yes, that’s it. No line numbers, no stack trace, no context. Errors in Go don’t have automatic stack traces like exceptions in Java would, as collecting the stack trace would cost resources.

However, we can fix this problem:

func GetPriceList(
    url string
) (PriceList, error) {
    result, err := Call(url, ...)
    if err != nil {
        return nil, fmt.Errorf("failed to fetch price list (%w)", err)

This will create a wrapped error with the following message:

failed to fetch price list (url cannot be empty)

The original error can be extracted from the returned error using the Unwrap() method, giving us a good way to handle context. We can also include additional information in the error by implementing a custom error, just like we would with an exception.

You can see the problem here already: it requires discipline. Discipline is something that is often lost when deadlines loom over developer heads and features must be completed.

Error types ▲ Back to top

As mentioned before, it is possible to create custom error types by implementing the following interface:

type error interface {
    Error() string

Simple enough? Sure, let’s do that:

type myError struct {
    message string
    someContextInformation string

func (m *myError) Error() string {
    return m.message

func (m *myError) SomeContextInformation() string {
    return m.someContextInformation

Cool! So, now we can return this custom error type, right? Like this:

func myFunc() *myError {
    return &myError{
        message: "my error happened",
        someContextInformation: "in myFunc",

Nope! This won’t work! You always have to declare the return type as error due to how errors are implemented under the hood. In other words, if you want the context information you have to perform type assertions like this:

err := myFunc()
ctx := err.(*myError)

Again, a bunch of boilerplate code. Finally, I settled on creating my own logging and error handling library.

Logging ▲ Back to top

Speaking of logging… Holy mackerel. So, Go has a built-in logging library. This logging library has two ways of using it:

  1. Procedurally, in a global scope with log.Print()
  2. By using the Logger struct

Both versions are different kinds of terrible. The procedural version introduces global scope into your program. In other words, good luck ever trying to run two tests in parallel and logging the output with the correct test.

The Logger struct on the other hand is not an interface, it’s a concrete implementation. If your code depends on this struct you are tightly coupling your code to an implementation of how log messages are processed. Sure, you can redirect the output, but not before the Logger munches on your log message a bunch.

To address this, a litany of logging libraries have popped up on GitHub, many of these take a procedural approach, most shamefully Kubernetes’ own klog. (Again, good luck with running tests in parallel.)

To make matters worse, any library you include may decide to write logs via the aforementioned wonderful logging libraries and there isn’t a thing you can do about it.

My solution? Drop any library that insists on logging itself without providing a useful interface and hide my logging solution behind an interface that all my code can depend on. Sad, but that’s the only way to do it.

Building and releasing ▲ Back to top

It may seem I’m taking a giant dump on Go, but it really isn’t all terrible, quite the contrary. It for sure isn’t the silver bullet many would claim it to be, but it’s a very workable and useful language. Sure enough, there are several shortcomings, but one area where Go really shines is building and releasing your application.

Go itself can cross-compile to a vast array of operating systems and CPU architectures by simply providing the appropriate build flags. The resulting binary is self-contained and doesn’t require external DLLs, SO files, etc (apart from the basic libc on Linux, etc). You can give someone a download link, and they will be able to run your application no matter what.

The Go ecosystem also has a wide array of build tools, one of the most well rounded being Goreleaser. These tools make it a breeze to build and ship your application in anything from OS packages to container images.

Libraries ▲ Back to top

Being able to package your application isn’t all that useful without an ecosystem of libraries you can rely on to get low level functionality working. Thankfully, Go has a vast array of libraries for all kind of server- and cloud-related tasks.

Some of these libraries are better, others are worse. You’ll need to do your due diligence in checking them out before use. If nothing else, the extensive standard library will keep you entertained for a while.

Concurrent programming ▲ Back to top

Go partly became notorious for its way of handling concurrency, from goroutines to channels. However, after having written this much code I rarely even pay attention to these anymore. I use them when I need them and other languages have awesome tools to deal with the concurrency too. Most applications I tend to see don’t even use concurrency because they are so small and simple that they don’t need it.

Should I learn Go? ▲ Back to top

Looking at the search terms of my previous two articles on Go I get a feeling that many people only visit these articles to get a confirmation on their already existing opinion.

There are certainly many reasons to hate Go, from the unusual syntax to the half-arsed way they decided to implement the programming model. On the other hand you can go all fanboy over how cool Go is because it makes it so simple to write microservices. (No, it doesn’t. It really, really doesn’t if you want to do it properly.)

If you jumped on either of these bandwagons, I would recommend getting off as soon as possible. Neither of these approaches are very professional and are telling signs of inexperience in software development.

Instead, judge a language by its usefulness for the task at hand. Are you trying to write a webshop? Try PHP or Javascript. How about an SMTP server? Sure, Go is a good choice. Machine learning? Maybe pick Python instead. ERP system? Java or Kotlin are a good choice.

For me, personally, Go is a very useful tool in my arsenal to write system-level applications such as ContainerSSH. It also replaced shell scripts in many areas.