Errbag is a Go package that allows Go programs to pause for a predefined amount of time when too many errors happen. It was written for crawld and made available as a separate package with a dedicated repository on GitHub.
You can skip directly to the next section if you do not care, as this part is not really important. ;-)
During the past year, I have been working on the DevMine project. One of the things we did was switching from Python to Go as the programming language for the implementation, with, in my opinion, great success. Of course, in over a year we did much more than that.
In the original Python implementation, we had a (very) simple GitHub crawler. In Go, this simple Python script turned into a rather complex program to crawl GitHub and clone git repositories, namely crawld.
Last June, I wrote a lengthy article explaining all the improvements we have made on the DevMine project and part of the work has been dedicated to crawld. We tuned this tool for performance improvements but also for better error handling. As this program was created to be running continuously, we need it to handle errors rather well. However, we quickly faced major problems when an unusual amount of errors were raised in a short time. This happened once with the network being down on the host machine, rendering crawld pretty much useless but also logging a lot of errors… Something had to be done to be able to deal with this kind of situations and this is why errbag was born.
Errbag is an error throttler. This Go package is meant to be used to record errors as they arise and throttle the operations when too many are recorded. The idea behind this is that a certain amount of errors is considered unavoidable (especially when dealing with external data) but when too many arise, something not good is happening and has to be dealt with.
So errbag allows some errors to go through but pauses for a
pre-defined amount of time when too many come in. This gives the sysadmin some
time to deal with the origin of the problem (like restoring the network or
freeing some storage space). As the
Record method can take an optional
callback function, you can even imagine something like sending an email when
Internally, errbag uses a channel, where errors are put in, and a function which takes errors out of the channel on a timely basis. To tune errbag, the size of the channel and the leak interval may be tweaked in order to allow more (or less) errors through before pausing.
Using a Go channel makes the whole implementation rather easy. Without comments, the whole code is around 80 lines (without the tests). I have been using Go for less than a year but to me, this is a perfect example of an interesting thing that the language makes rather easy to implement.