What keeps us Go'ing

A while ago I had to opportunity to give a lightning talk at the Sydney Go meetup. Go is increasingly being used at Bigcommerce, especially within the Data team, and I'd like to outline with some examples why we like and use Go. First, check out the slides, and then I will expand on some of the more interesting points.

We've been using Go at Bigcommerce for over an year now. We have built an event collector, that acts as an API endpoint to which anyone can send event data to. This system accepts, validates and stores events, and currently processes around 8 billion events a month, with close to 100% uptime. Go is the first language we consider for all new data projects. Here's why.

Concurrency

Go has goroutines and channels that make building concurrent programs simple. Goroutines are like lightweight threads, and channels are typed queues that allow allow different parts of the program to communicate.

We create a (fixed size) channel by:

Channel = make(chan Type, int(Size))

and the producers can enqueue messages on the channel using:

select {
    case Channel <- Message:
        // Successfully queued message onto Channel
    default:
        // Queue full, and message could not be enqueued
    }
}

To complete the picture, multiple goroutines can read from the same channel by:

select {
    case Message <- Channel:
        // Process message here
    }
}

And so, we have just broken down our data processing pipeline into multiple stages, each of which can be parallelised independently, all without using error prone concurrency primitives like locking, mutexes or having to declare atomic/volatile types.

HTTP/JSON

JSON is the lingua franca of the modern web, and Go has very good support for it. Consider:

type userPayload struct {
	UserId      string `json:"userId,omitempty"`
	AnonymousId string `json:"anonymousId,omitempty"`
	OriginIP    string `json:"originIP,omitempty"`
	Useragent   string `json:"userAgent,omitempty"`
}

where we define a type, and how it should be unmarshalled from a JSON string.
When you want to express the idea that the userPayload is an expected part of various request types, Struct Embedding allows us to easily do so:

type Foo struct {
	userPayload
        ...
}

type Bar struct {
	userPayload
        ....
}

and we don't need to repeat any of the marshalling/unmarshalling code.
We also note that creating web servers in Go is similarly simple and there exist thousands of examples on the internet already.

Testing

Go's testing support makes it very clear that the language designers didn't just add support for testing as an afterthought, but designed for it from the beginning

• Testing is part of the stdlib - testing
• The go tool understands and makes it easy to run all tests or any subset thereof:go test
• Interfaces that are implicitly satisfied make it easy to code against an interface, and not an implementation, and
• Performance testing is baked in: http://dave.cheney.net/2013/06/30/how-to-write-benchmarks-in-go

Performance/Great compile times

Prepare to be blown away by the CPU and memory utilization of your typical go program compared to say, an equivalent python or ruby program. A typical go project compilation takes 2-5 seconds. We're talking thousands of LoC. And even though Go is garbage collected, it doesn't feel like that coming from Java - a typical Go program matches Java in performance while taking way less memory and way fewer GC pauses.

Deployment

Since Go programs compile down to a static binary, deployment is usually some version of "scp, symlink; run". Painless deploys, especially compared to Python, Ruby or JS, will make your ops team love you. In most startups, the ops team is basically just you, so this is personal at least some of the time.

Tooling

The tooling thats included with Go makes it easier to write clear, readable, and correct code -

• gofmt is a consistent way to indent code that enables readability and eliminates wasteful debates
• govet catches many errors before they are deployed
• go race helps catch race conditions in code
• go get to easily pull in dependencies
  ... and many more to list

Conclusion

For backend code, Go will allow you to put working solutions in front of your customers faster than any other language I've used. Its also an extremely simple language to pick up - any developer should be able to become productive in go within a month.