[This post has been translated to Russian by Vlad Brown.]

Channels and goroutines form the core of Go’s CSP-based concurrency mechanism. Read on to pick up some tips and tricks about channels, specifically “buffered” channels, that are commonly used as queues in a producer-consumer context.

Buffered Channels = Queues

Buffered channels are first-in first-out (FIFO) queues of bounded capacity. The capacity is fixed at the time of creation of the queue – queues cannot be resized on the fly.

queue := make(chan Item, 10) // queue with a capacity of 10

Each item in the queue can be up to 64KiB in size, and may or may not contain pointers. If you do have pointers, or if the item iself is a pointer, it is up to you to ensure that the pointed-to objects remain valid while it is in the queue and being consumed.

queue := make(chan *Item, 10)
item1 := &Item{Foo: "bar"}
queue <- item1
item1.Foo = "baz" // valid, but not a good idea!

Producers (code that pushes items into the queue) can enqueue items with or without blocking:

// next line will block queue is full
queue <- item

// for a non-blocking push, do this:
var ok bool
select {
    case queue <- item:
        ok = true
    default:
        ok = false
}
// at this point, "ok" is:
//   true  => enqueued without blocking
//   false => not enqueued, would have blocked because of queue full

Consumers typically pop items off the queue and process them. If the queue is empty and the consumer has nothing better to do, it can block until a producer pushes an item.

// next line will pop an item, or wait until it is possible to do so
item := <- queue

If the consumer should not wait, use this:

var ok bool
select {
    case item = <- queue:
        ok = true
    default:
        ok = false
}
// at this point, "ok" is:
//   true  => item was popped off the queue (or queue was closed, see below)
//   false => not popped, would have blocked because of queue empty

Closing Buffered Channels

Buffered channels are best closed by producers. The channel close event is signalled to the consumers. If you need to close the channel from outside of the producer or the consumer, you must use external synchronization to ensure that the producer does not attempt to write to a closed channel (this will cause a panic).

close(queue)  // closes the queue

close(queue)  // "panic: close of closed channel"

Reads and Writes to Closed Channels

Can you write to closed channels? Nope.

queue <- item // "panic: send on closed channel"

And reads from closed channels? Actually, before scrolling down, guess what this program outputs:

package main

import "fmt"

func main() {
    queue := make(chan int, 10)

    queue <- 10
    queue <- 20

    close(queue)

    fmt.Println(<-queue)
    fmt.Println(<-queue)
    fmt.Println(<-queue)
}

Here’s a playground link.

Surprised? If you are, remember you read it here first! :-)

Reads on closed channels behave differently:

  • If there are items yet to be popped off, the popping off happens as usual.
  • When the queue is empty and closed, the read will not block.
  • Reads on empty and closed channels will return the “zero-value” of the channel item type.

That should let you figure out why the program printed out what it did. But how do you tell if the read was valid or not? The zero-value might be valid, after all. This is the trick:

item, valid := <- queue
// at this point, "valid" is:
//    true  => the "item" is valid
//    false => "queue" was closed, "item" is only a zero-value

This let’s you write consumers like this:

for {
    item, valid := <- queue
    if !valid {
        break
    }
    // process
}
// at this point, all items ever pushed into the queue has been processed,
// and the queue has been closed

In fact, the “for..range” loop is an easier way of writing this exact same thing:

for item := range queue {
    // process
}
// at this point, all items ever pushed into the queue has been processed,
// and the queue has been closed

And finally, we can also combine the non-blocking and valid checks into one:

var ok, valid bool
select {
    case item, valid = <- queue:
        ok = true
    default:
        ok = false
}
// at this point:
//   ok && valid  => item is good, use it
//   !ok          => channel open, but empty, try later
//   ok && !valid => channel closed, quit polling

Go Consulting & Training

Need help getting a project off the ground using Golang? We have extensive experience in creating and running production grade Go-based software solutions. We can help you architect and design Go-based projects, or provide advice and mentoring for teams that work with Go. We also provide training for teams looking to get started with Go or to extend their Golang knowledge. Find out more here or contact us today to discuss your requirements!