Lately I've been doing more work in the Go programming language. Today I thought I would share three "gotchas" that caught me off guard, or otherwise produced results that I would not have expected in my work with Go.

1. The range clause

The range clause is very convenient. It allows you to iterate over a slice or map with two variables which represent the index and the value of each item. For example:

for index, value := range mySlice {
    fmt.Println("index: " + index)
    fmt.Println("value: " + value)
}

However, something notable is happening under the hood. Let's see more complex example:

type Foo struct {
    bar string
}

func main() {
    list := []Foo{
        {"A"},
        {"B"},
        {"C"},
    }

    list2 := make([]*Foo, len(list))
    for i, value := range list {
        list2[i] = &value
    }

    fmt.Println(list[0], list[1], list[2])
    fmt.Println(list2[0], list2[1], list2[2])
}

In this example we are doing a few things.

1. We are creating a slice of Foo structs called list.
2. We are defining a second slice of pointers to Foo structs called list2.
3. We iterate through each struct in list in order to assign its pointer to the corresponding index in list2.

So therefore you might expect the output of the above code to be the following:

{A} {B} {C}
&{A} &{B} &{C} 

However, this is not what is happening. Let's take a look at the output of this code:

{A} {B} {C}
&{C} &{C} &{C} 

The first line is as expected. These are the structs we initially created in list, but the second line is unexpected. It looks like we are printing out a pointer to the last struct in the list three times. But why is this happening ?

The culprit is the range clause.

for i, value := range list {
    list2[i] = &value
}

Here's the problem: Go uses a copy of the value instead of the value itself within a range clause. So when we take the pointer of value, we're actually taking the pointer of a copy of the value. This copy gets reused throughout the range clause, which leaves our list2 slice full of three references to the same pointer (the copy pointer).

According to the Go reference manual, "The iteration values are assigned to the respective iteration variables as in an assignment statement." So effectively you can imagine the above range clause to be the same as writing:

var value Foo
for var i := 0; i < len(list); i++ {
    value = list[i]
    list2[i] = &value
}

In order to produce the output we expect, we should use the index to take a pointer to the actual value, instead of the copy.

for i, _ := range list {
    list2[i] = &list[i]
}

2. The append built-in function

Slices are primitive types in Go. In other languages you might reach for an Array where in Go you would reach for a Slice. Here's an example of how we might add a value to the end of an integer slice.

list := []int{0,1,2}
list = append(list, 3)
fmt.Println(list) // [0 1 2 3]

On the surface it seems to be similar to a push() array method, but slices are not quite arrays, and the built-in append function surprised me with its behavior under the hood. Have a look at the example below:

func main() {
    a := []byte("foo")
    b := append(a, []byte("bar")...)
    c := append(a, []byte("baz")...)
        
    fmt.Println(string(a), string(b), string(c))
}

In this example we define a slice a of bytes with an initial value of ["foo"]. Next we append another slice ["bar"] to our initial slice a, and again we append another slice ["baz"] to our initial slice a. The output of the above snippet is:

foo foobaz foobaz

Whaaat? 😮 Shouldn't it be foo foobar foobaz ?

In order to understand what's going on here we have to understand what a slice really is. A slice is a descriptor which consists of three components:

1. A pointer to an underlying array - that is, one allocated by Go which you don't have direct access to.
2. The capacity of said underlying array.
3. The effective length of the slice.

So what's really happening ? Go will reuse the same underlying array in append() if it can do so without resizing the underlying array. So all three of these structs are referencing the exact same array in memory. The only practical difference is their length value which in the case of a is 3, and in the case of b and c is 6.

Keep in mind, Go will only reuse the same underlying array if the length of the newly created slice is less than or equal to the capacity of the initial slice.

It's important to understand what is happening under the hood when using some of the built-in slice functions. For more info, Robe Pike wrote a very helpful blog post which goes into a lot of useful details around slices.

3. Variable shadowing

When people first look at Go code for the first time, one of the first things they ask is "What is that := operator for?". Well, as you know, it's a shorthand variable declaration operator. It's used for both declaring and setting the value of a variable. The type is implicitly declared. This is very convenient, but can lead to some problems if you aren't careful. Specifically I'm talking about variable shadowing. This is something that has bitten me in particular, because I have been programming almost exclusively in es5 javascript for the last two years where there is no block-level scoping.

Check this out:

func main() {
    list := []string{"a", "b", "c"}
    for {
        list, err := repeat(list)
        if err != nil {
            panic(err)
        }
        fmt.Println(list)
        break
    }
    fmt.Println(list)
}

func repeat(list []string) ([]string, error) {
    if len(list) == 0 {
        return nil, errors.New("Nothing to repeat!")
    }
    list = append(list, list...)
    return list, nil
}

This is somewhat of a contrived example, so bear with me because I think it illustrates the point well. The run down of this snippet is as follows:

1. We create a slice of strings, list.
2. We enter a for loop.
3. The for loop calls a function repeat() which returns a new slice, and an error.
4. We break out of the for loop, and we print the value of list.

You might expect the output here to be:

[a b c a b c]
[a b c a b c]

but in fact, it is:

[a b c a b c]
[a b c]

Because we use the shorthand variable declaration operator, we are actually redeclaring the list variable inside of the scope of the for loop. This is awkward, because the second variable err is a new variable which we want to declare. We can fix this by changing the first few lines within the for block to:

var err error
list, err = duplicate(list)

One way to catch these problems early is by taking advantage of the go vet tool. It has an option -shadow which can help to detect issues with variable shadowing.

There is a good blog post called The Golang Beartrap that talks more in detail about variable shadowing.

Summary

Go is a great language, and once you understand its quirks it can be a real pleasure to work in. I really appreciate the fantastic tooling, and readability of the language. Good luck, and happy gophering! 😄