Navigation

Wild Wild West (or, Thoughts On Being An Effective Sheriff)

Monday, July 30th, 2007

Last week Wednesday it was my turn to be the Mozilla sheriff. It also happened to be the last day to check in code changes for Firefox 3 alpha 7. Needless to say, it was a hectic time, but I think I learned a lot of good lessons about how to be effective as a sheriff. Here’s my advice, for better or for worse:

  • #developers is your friend. I can’t imagine trying to sheriff without the #developers IRC channel on irc.mozilla.org. It should be your primary source of communication with others, if possible. One of the greatest things is that there are lot of other folks there, and nearly all of them are interested in build bustage as well.
  • Delegate, delegate, delegate. Don’t get caught in the trap of trying to fix (or even interpret) build bustage on your own. If at all possible find someone (preferably the author of the patch) to identify and fix the bustage, and check back with them to make sure they follow through. Meanwhile, keep your eye on tinderbox for additional fallout.
  • Refresh tinderbox.mozilla.org like a fiend. While firefighting, I refreshed it pretty much every time I looked at the page or someone mentioned something happening in #developers. You can refresh less often when things aren’t in the middle of landing, but it’s still a good idea to be relatively on top of the build. Bottom line, I don’t find the auto-refresh on tinderbox to be nearly frequent enough, so manual refreshing is a must.
  • Have something else to do to occupy your time, but make sure it’s something you can easily drop and come back to later. Even better if it has regular pauses where you can check the tree. For example, I had the some TV on. When the show went to a commercial break or an episode ended, I’d refresh tinderbox. I did a little bit of hacking when things slowed down, but I spent most of my day fighting fires and trying to figure out why certain tinderboxen just wouldn’t build properly, so I was pretty busy. YMMV.
  • Don’t be afraid to close the tree. It’s not something to do lightly, but when things start to get crazy and you’re trying to make sure the build goes green, go ahead and close the tree. Make sure to update the tinderbox status message though, so people know why the tree is closed and when they can expect it to re-open.
  • If at all possible, attempt to coordinate with people who are going to be landing big changes. #developers is an excellent resource. Just ask if anyone is planning on landing anything big, you’ll usually find out what’s probably going to land that day.
  • Make sure either you or the person who checks in a bustage fix stars the tree in a meaningful way. It’ll save you a lot of pain — knowing that the starred build or the build after the star is supposed to go green means you spend that much less time staring at tinderbox and more time doing other things. It also lets others know that progress is being made and they can get ready to check in if they’re waiting.
  • Look at the performance graphs and other numbers, but don’t spend too much time trying to analyze them. In my experience, at least, the graphs are generally really noisy. It’s pretty clear after a couple of hours though if there’s a regression, though, so checking these two or three times during your stint as sheriff is probably enough.

I’m going to go out on a limb and say that my day was more frenzied than most, but it definitely wasn’t horribly abnormal. The upshot of all the above advice is that as sheriff, your primary responsibility is to monitor the tree and make sure that the right people know when things break. Don’t sidetracked trying to understand how or why tinderbox does the things it does. Just make sure that progress is being made and that the right people have been alerted.

Take everything I’ve said here, of course, with a grain of salt — I’m still fairly new at sheriffing and I’d love feedback if anyone thinks what I’ve said is wrong or has any additional suggestions.

Comments

  1. asa@mozilla.org replied on July 30th, 2007:

    Great write-up, Colin. It’s been 5 or 6 years since I sheriffed but it obviously hasn’t changed much :-)

    All of your points are really strong and this could actually be added to the sheriffing document (if we have one still)

    • A
  2. dmose replied on July 31st, 2007:

    As one of the participants in that exciting after, I think you did a great job.

    Dan