On April 6, I hosted the 3rd session of a monthly Company Building webinar series for founders and operators of developer-focused and/or commercial open source startups. The goal of this series is to share tactical knowledge -- the nitty gritty know-hows that can make the grinding experience of company building a little easier for entrepreneurs -- from experienced operators who are still in the game. (Our 4th session is on June 6, 4PM US Pacific Time. Discussion topic: Developer Relations best practices with Kevin Chen, the 1st Developer Advocate at Kong. Please register HERE.)
This session is with Justin Cormack (CTO of Docker) on the best practices of managing and scaling a global engineering team.
To make our wide-ranging conversation more useful and digestible, I edited and condensed the session’s transcript into the following topics (jump to the section that's most relevant for you):
- Justin's entry into tech
- How Docker became a more distributed company
- Best practice: coordinating engineering by time zones
- Ideal team size and synchronous vs asynchronous work
- Collaboration between engineering and other company functions
- One mistake Justin hopes all entrepreneurs can avoid
You can find the audio/video version of this conversation on the Interconnected Voices YouTube channel or watch it below:
Be sure to check out transcripts and audios of Session #1 with Matt Ekstrom (ArangoDB, ScyllaDB, MongoDB) on early stage sales and Session #2 with Clint Smith (Discord, Datastax, MySQL) on global company operations. Bookmark the "For Founders" tag for future content.
Kevin: Justin, I really appreciate you taking the time. I know it's late into your day in the UK, but thank you for joining us.
Justin: Thanks for having me. It's really nice to see you again. I really love your weekly emails. Interconnected is one of my favorite things to read and I always look forward to reading it. It's really, really interesting. And there aren't other people writing with the same kind of breadth and it's one of the things I really appreciate. I guess I've got a weirdly broad background too, in many ways, so it kind of resonates with me.
Kevin: Well, thank you so much for that, Justin. I appreciate you giving Interconnected a shoutout, but today it's all about you! So let's actually start with that background. I know you've been at Docker for many years, both as an engineer and now rising up to become its CTO, but I'd love to hear, first of all, your pathway into technology. And then we can dive into some of the engineering management topics that we will talk about.
Justin: So I think my route to technology was fundamentally a lot tied to my mother. She was a math teacher and always loved helping us understand things. She did computer programming a long time ago when she was younger, so she introduced us, me and my sister at an early age to computers and technology. And that's kind of where it came from, but I was never quite sure what I wanted to do. I always had broad interests, so I ended up tangentially leaning towards technology, but not jumping into it full time. I worked in finance for a while and investment management, but gradually moved more and more towards software.
I’ve always been interested in business and what’s going on there as well, so I did a few startups. I did startups in IoT, LED display, public advertising a long time ago. I ended up getting interested in Docker indirectly, because I got interested in working on unikernels, which are basically trying to encapsulate operating systems at a more developer focused level. And we did a startup, Unikernel Systems, that was acquired by Docker five and a half years ago now. So that's how I ended up at Docker.
Docker is a really fascinating place. The people are great. I always like working there. It's a really interesting thing from, you know, how do we take open source and turn it into a business? How do we take something that's got mass adoption and created a whole market, and turn it into a business, which everyone else feels they can sell off of as well? And it's been a really interesting journey through that.
So I started off working as an engineer based in Cambridge, UK, which we'll talk about the international things a little bit later on, but basically I worked on Docker desktop. That was our first project we did, which was great fun. We shipped that so it became really successful.
And I evolved through lots of different roles in Docker, working across a lot of different parts of the business. As you mentioned in the intro, I worked in security for a while. Security was always something I'd kind of worked on the edge of, but hadn't really jumped fully in. And it was really interesting to work on that. I mainly focused on product security, but really running all of the security at Docker.
Then Docker restructured at the end of 2019. We sold off the enterprise Kubernetes business to Mirantis and focused really on the roots of developers and developer needs and the initial adoption of Docker, which was all through developers. In that role, I was really the most senior engineer at Docker. By then I kind of started moving towards this sort of CTO role. At the end of the year, I spent a lot of time talking to our investors, Peter Fenton, in particular at Benchmark, and talking about the directions that were interesting and what we need to do. So that's how I ended up moving into the CTO role early this year.
Docker’s Journey Becoming A Globally Distributed Team
Kevin: That's fascinating. I want to actually start with your beginning journey into Docker then. I didn't know that you came into Docker via an acquisition and that's kind of interesting. This was back in 2015 or so, is that right?
Justin: Yeah that's right.
Kevin: So back then Docker was an American company but with some European roots, right? Its founders are French and so on. And you have always been based in the UK. What was that initial, entry way into working remotely or in this globally distributed engineering setup as a acquiree of an acquisition. Would love to hear more about that experience.
Justin: Yeah, I think it was really interesting and if you follow on from there, it's affected a lot of how Docker changed down the way in terms of international things.
So we had the advantage of being quite a big team. I think there were 10 of us at the time. So it wasn't a sort of two-person acquisition or anything, and that gave us a bit more leverage. We were quite clear from the beginning that we didn't want everybody to move to SF. For pretty much all the previous acquisitions except one, everyone had moved to SF.
So we were a sufficient nucleus that was the beginning of Docker's real international spread. It was kind of fascinating though, because I remember at the time they really weren't sure how this was going to work out. And I think we actually had in our contract that said that someone from the team should be in SF at all times. That was sort of a hedge, like you’ll be in America, but that’s not how it worked out. Until COVID, I spent a lot of time in the US. I probably spent more time than the other people. I used to travel every two or three months to the US and spent a few weeks there.
At the time Docker was still mostly SF but that has changed over the years. You mentioned Docker was initially a French company and so fairly soon after we were acquired, in maybe late 2015, or early 2016, we set up a Paris office. We've always found it easy to hire people in France because Docker has this reputation of being a French success story, which is really interesting. So we've had a lot of people in the Paris office who were initially in SF. It used to be quite French speaking on a Friday afternoon in SF: you'd hear a lot of people speaking French, that kind of thing. But now most of the French people are at the Paris office. That was the beginning of really becoming a globally distributed company.
Best Practice: Coordinating Engineers By Time Zones
Kevin: I see. So let's get into that then. It sounds like you, or your previous company was the reason why Docker became distributed, almost out of this acquisition. And now it's becoming more popular, more generally practiced, or even fashionable these days for new startup companies to just have a corporate entity somewhere but you hire from wherever. GitLab, which is a company we're both very familiar with being kind of the prototypical example of that in some ways.
How has your engineering management practice evolved since you've risen up the rank? What do you do to manage all these engineers from all time zones who may even speak different languages, but are all working from the same code base or computer language? What are some best practices you've gleaned so far?
Justin: So it's evolved a lot over this period. I think we kind of had to learn to do it, and we weren't very good at it at the start. Also there's a big change pre and post COVID. Until COVID we were just starting to become a remote work company.
Some people have always been remote at Docker, but it was a small number and we were starting to have a lot more remote working. We had the SF office and a Palo Alto office, and people were moving to whichever one was more convenient or working from home. More people even before COVID were moving away. And we were hiring people outside the offices.
But when COVID happened, we just got rid of our SF office after the Mirantis split, because we were downsizing. And then we just talked to staff about offices and working and gradually people came around to getting used to working from home. And we shifted now into a situation where we're just rolling off the last places for offices and going fully remote. So there were two transitions really, one was to distributed and one was to fully remote.
So the distributed one culturally felt a bit different because we still had offices and we still had these clusters of people always with local management. So we'd have engineering management in Paris, engineering management in Cambridge, engineering management in SF. And there'd be this kind of close knit teams generally working together.
We had times when some of our teams were very split between people mainly working across time zones and not with people in the local time zone or office, which was very weird. And we've really gravitated towards the model where we try and keep coordination within particular time zones but also ideally local offices. I think we're now more focused on the time zone as a unit rather than a place, but time zones are difficult. France to the West Coast is even worse than the UK to the West Coast - it's another hour off. And it's difficult for people to spend enough time together, to work really, really well together. It can work and open source projects often do that, but coordinated working on individual close-knit code bases and products is really hard.
So we're doubling down on that small connected team. We're moving now to teams that are very self-contained, ideally geographically localized, maybe around five engineers, a product manager and engineering manager, and UX. Those small teams are ideally in a single time zone. Really aligning them to solve particular customer problem areas.
Kevin: Got it. I mean it is kind of a wild, wild west. I don't think anybody really has this thing down as far as, “this is how you're gonna manage 100 or 200 engineers scattered across the globe.” A lot of companies are figuring it out as they go. And it sounds like the way you're approaching it now is actually using time zone as the unit. So if I understand it correctly, is it that you will intentionally cluster say a few engineers that need that close knit work or in a particular unit to be in the same time zone, so they can coordinate in a more synchronous fashion?
Justin: Yeah, absolutely. So far we've been restricted in which countries we're hiring, because we were limited by where we had companies. We're going to move to less restrictive hiring and use a global hiring company so we can hire anywhere now, but we were kind of stuck.
We were across the US and Canada. We're in the UK and France, which are an hour off, so pretty much the same time zone. I think we're going to be cautious in expanding the time zones above those areas for now, until we're larger. We already refocused Docker into 70 people now. So keeping that small, tight knit thing is important. I think it'll be a little longer before we're going to move into the Asian time zone probably, but we're going to expand the countries we hire in the existing time zones we have, which is a fairly large block of the world at the moment.
Ideal Team Size; Synchronous vs Asynchronous Work
Kevin: Totally. So to get really tactical now. Based on your experience so far, organizing what I would call engineering pods or small engineering teams along the same time zone, how many engineers have you discovered is the ideal size for a pod like this to work on something together where synchronous work again is a preferred way to approach it?
Justin: Around five engineers, plus the engineering manager, product manager, and UX. We've had larger and smaller teams before, but that seems to be kind of optimal where we're trying to aim at at the moment and see. We're really trying to get the focus where the team has all the resources it needs. We've had central functions for design before, but we've had bottlenecks with central functions and we'd like to really get every team their resources.
Kevin: And what's your opinion now on this balance between synchronous work and asynchronous work? These terms get thrown around a lot, about engineering practices, especially in open source companies. And for those of you who aren't too familiar with the jargon of it all, but if you do any programming, you will know.
Asynchronous just means somebody does it somewhere and then it can wait and somebody else can pick it up somewhere else. And you can do this without really coordination at the same time, while synchronous is the opposite, where you do want to talk through certain things and work on something together at the same time to get the results.
What's your thought on the balance between these two approaches in terms of growing an engineering practice that serves the rest of the company well?
Justin: Yeah, it's difficult because there's a lot of different kinds and forms of communication. The traditional open source model of mostly communicating through issues and pull requests is very async, with perhaps some meetings to set direction but not very often. This tends to be very different from the product-led thing, the pair programming and sprints and stand up meetings and that kind of close-knit communication. So I think there's different kinds of communication and models and they work for different things at different stages.
We are an interesting company in that we have a lot of different kinds of work we're doing and software we're working on. We have open-source work that's in the community and fits more with the very async model. And then we're running SaaS services where people are on call, which is a different sort of communication.
We're kind of unusual in the variety of things we do as a company, which makes it more interesting, but also makes our communication structures a little bit different. I will say getting people better at writing things down is incredibly valuable, whatever you're doing, and writing is in a sense async. But it’s not necessarily about what form of async communication you use, but how well you're doing it. I'm a big fan of the Amazon style writing things down before meetings. We've been moving well towards that culture. We had a bunch of people who are ex-Amazon, who helped introduce those writing cultures to us, which is really valuable.
It's interesting when you can change from presentation/slide cultures to writing cultures over time. And I think being fully remote has changed things as well, because lots of people have white boarding and synchronous meeting cultures that are actually harder to do remotely. Whereas writing is actually a culture that works very well across different scales and sizes of communication. It helps people on board as well. So it has a lot of advantages.
Kevin: Absolutely, yeah. I’m clearly, as a newsletter writer, a big fan of writing. I do think it's probably still one of the most scalable forms of communication, that could both spread out very quickly, very easily, but also get a lot of people on the same page at different points in time. Like you said, probably one of the better, asynchronous ways of communication.
Justin: There is one thing about writing that's actually interesting. It's about language. So people who speak different languages generally often prefer reading to listening, because they can read it at their own pace, but people do find writing in their non-native language more difficult. So there are some nuances around that that are interesting.
Kevin: It's interesting you mentioned that reading is better, for say non-native English speakers, because you can take your time. You don't have to listen to somebody else talk about what we're doing right now, which is a lot harder to absorb in real time. Any advice or thoughts on making writing more accessible as a tool of expression within a company and helping non-native speakers of certain languages write better?
Justin: I think a lot of it is practice. I think writing things down in their native language first often helps people. I actually had an interesting conversation with someone at Amazon about this, and Amazon does not have any English-only rule with that writing policy - you could write it in any language. That has problems potentially too, but it’s an interesting slant on it.
One thing that I've come across with open source projects is that sometimes they're divided by language. I remember the first time, I was an early user of OpenResty, which is a lower, larger plugin for Nginx and that was originally in Chinese. And I used to read the mailing list with Google translate at that time. It was interesting because it gradually transitioned to more English. The author moved to America, but there were communities then that were divided by language. We’re still seeing that some of the cloud native projects a little divided by time zone and language. These things are just difficult.
Collaboration Between Engineering Teams and Other Functions
Kevin: I want to move on to a couple more topics before we open up to Q&A. I know we already have a couple of questions in the Q&A tab here in Zoom. One thing I want to talk to you about is, of course, Docker is a very technical company. The product is very technical, but obviously engineering does not live in its own world, right? It has to work with the product team, the design team, and also the sales and even probably a marketing function as well. What are some best practices you could share with our audience in terms of modes of collaboration between your engineering teams and these other functions in the company, especially in this distributed or multi-time zone world?
Justin: Yeah, I think it's been interesting. Docker has changed a lot and these practices have changed a lot as well. We went from being in the enterprise market to developer focused.
I think the engineering team’s collaboration with sales, marketing, and product are very different. With marketing, we've had this amazing community that's built up really earlier with Docker and our products. And marketing has been around growing that community and building events like DockerCon that are amazing. We had 20,000 sign ups just in the first few days when we opened the registrations for DockerCon this year last week. We had 80,000 people last year at DockerCon. These events are what people really value and a lot of our marketing is about nurturing those communities that we've built up around the products.
The engineers love to get involved in that and people really love communicating with engineers and spending time with them, because they're the people who build these tools that they love. And so the interaction between the community and marketing has always fallen out quite naturally.
Sales has changed more in a way. We had a full, large enterprise sales force when we were in the enterprise Kubernetes and runtime business and that was very much me going to meetings and talking to banks about security and things like that, and shipping relatively slow-moving enterprise products that support the enterprise sales process, so engineers often feeling quite detached from sales. It's more difficult to involve them with that. And it's just that engineers don't like enterprise sales, because they don't like being sold to like that themselves most of the time.
Whereas now we have a tiny sales team and we're mainly bottom up SaaS. We're re-exploring what sales is like for a bottom up developer focused company. So it's a totally different relationship we have with sales now.
There was a massive disconnect with Docker, because the grassroot's adoption of Docker was people who are moving to cloud early, but we were trying to sell products to people who were trying to do on-prem and moving into cloud late. It was a kind of mismatch in the end. I think we're in a great situation now where we're really just engineers trying to sell to people like themselves. We're building products for our peers. Developer tooling companies have this amazing thing, where you are building things for yourself and can say that these people are like me. They're not necessarily exactly like me - they might be less advanced in this journey - but there's much more empathy with them, and that makes it easier.
I think from the product point of view, that also helps. I think that, yes, our products are technical. And Docker's original thing was to take something that was deeply technical and obscure - Linux namespaces and containers - and turn them into something that everyone could use. But the market and the users of Docker have changed so much in the five years I've been at Docker. I realized that now they don't understand how to use SSH anymore.
We have over 10 million people using Docker, and it's all sorts of developers at all sorts of levels of experience and technical knowledge. Most of them don't understand the inner workings of how it is. It's just magic for them. And that's very different from people who had opened pull requests to fix an issue with the Linux namespaces five years ago.
So I think it's hard for product people to work with these things, because it is deeply technical. And I spend a lot of time as CTO working with the product managers to help them understand what's possible. And I'll listen to the engineers do that, help them understand what's possible and help them interpret some of the things that they're saying in terms of what users are asking for and thinking about the possibilities.
But there are product people out there who do work and their careers are on developer focused products. User experience for command line is still a niche area, but there are people who are fascinated by it because it's just an interesting topic. It's understanding the modern world. And what developers do now, other people will be doing in the future. So it's a fascinating area to work in product and we're hiring product managers, by the way, if anyone's interested.
Kevin: Never miss the opportunity to put out a recruiting pitch, right? We're all in tech. Everybody is hiring, but yeah, I think you're right. Docker has probably done almost too good of a job back in the days to obscure all these details in the kernel space and on the Linux level of things so that people are like, I don't need to know this. I would just use Docker, right?
Justin: But that's the right attitude I think. I sense that Linux isn't necessarily something you have to use. Sometimes the Linux desktop experience was based on the Sun Workstation experience and it was kind of nice back then. I remember the first time I sat in front of the Sun Workstation. I was like, "wow, this is amazing." But it's not actually necessarily the way people want to consume Linux now. I mean, people actually want to consume Linux as a deployment platform for applications, but Docker is a kind of an API that you can do that without having to use Linux. And the kind of way that people think of, it is like, "it's a programming API, so it doesn't have to be your life." You can still use a Mac or Windows. Developers think of their life in IDEs, in pull requests, and in code. The operating system doesn't have to be a really visible thing.
One Mistake Justin Wants New Entrepreneurs to Avoid
Kevin: Absolutely. So I want to wrap up with a broader question before we go into the Q&A session. Because you've been through a lot of startups before Docker, and now you're at Docker and you're the CTO. I'd love to hear maybe one mistake that you hope all new entrepreneurs, new founders in our space can avoid when they start growing and scaling their engineering team in the early stage.
Justin: I think that one day you'll have someone who doesn't fit and doesn't work in the organization. Donnie Berkholz, our chief of product, has a great talk he did many years ago at RedMonk about assholes ruining your open source projects. It also applies very much to companies that have people who are just negative and destructive of what you're trying to achieve. Recognizing these early and fixing the problem is really important.
If you're lucky it'll happen sooner and you'll get to learn that lesson sooner, but it will eventually happen at some point. There are people who just are not the right fit for your startup at the stage it is now. You just shouldn't be working with them.
You need to set the culture soon. One of the reasons I still love working at Docker is that the culture is just really nice. People are just lovely people to work with, very modest and great at working together. And we've felt that culture for a long time and it's been there since very early on.
One of the things that as a founder you bring to the company is those attitudes of how you want people to be and how you want people to behave to each other. For eight years now we've kept very much the same culture and quite a few of the same people have been there for a lot of that period too, not all of them. The culture has stayed longer than the individuals who helped create it.
If you like what you've read, please SUBSCRIBE to the Interconnected email list. To read all previous posts, please check out the Archive section. New content will be delivered to your inbox (twice per week). Follow and interact with me on: Twitter, LinkedIn, Clubhouse (@kevinsxu).
四月六号，我主办了每月一次的 “创业公司建设” 系列webinar的第三期，受众专注于面对开发者产品和/或商业化开源创业公司的创始人和运营者。 (第四场会在六月六号，美国西部时间下午4点开始。题目是：Developer Relations的最佳时间。嘉宾是Kevin Chen，独角兽Kong的第一个Developer Advocate。请在这里注册！)
这一场是与 Justin Cormack（CTO of Docker）讨论管理全球分布式远程办公的工程团队的各种最佳实践。
Kevin: 让我们欢迎Justin上台! Justin非常感谢你能抽出时间来，我知道你在英国的时间已经很晚了，但还是感谢你能来参加我们的活动。
我一直对商业领域也很感兴趣, 所以后来就做了一些创业公司。 我做过与IoT、LED显示屏和广告方面有关的各种创业公司。我最终间接地对Docker产生了兴趣，因为我对unikernels产生了兴趣，这基本上是试图在更多开发者关注的层面上对操作系统进行封装。
Kevin: 明白了。我想这确实是一个还在被开拓的领域，我觉得没有人真的摸索出了所有具体该怎么做的事情，能够做到 "这是你该如何去管理100或200个分散在全球各地的工程师的方法"，很多公司都还是在摸索阶段。听起来你现在的做法是以时区为单位，就像你说的那样。所以，如果我理解正确的话，是不是你们会有意地把几个需要紧密工作的工程师集中起来，在同一个时区，这样他们就可以以更同步的方式协调？
Justin: 嗯，这很难，因为有很多不同种类和形式的沟通。传统的开源模式主要是通过issues和pull requests进行沟通，是非常不同步的。也许会开一些会来确定大方向，但并不频繁。这往往和产品主导的东西，双人编程和sprint以及站立会议那种紧密沟通的协调方式有很大的不同。所以我觉得有不同的沟通和模式，它们在不同阶段对不同的事情有不同的作用。
Justin: 但我觉得这才是正确的态度。我感觉Linux不一定是你必须要用的东西，有时候Linux桌面体验是基于Sun Workstation的体验，那时候挺好的。我还记得自己第一次坐在Sun Workstation面前的时候，当时就想，"哇，这太神奇了"，但实际上这并不一定是现在人们想要用Linux的方式。我的意思是，人们其实是想把Linux作为一个应用程序的部署平台来用，Docker就是这种API，让你可以不需要使用Linux就能做到这一点。而人们对它的看法是，"它是一个编程API，所以不必成为你的生活的一部分"，你仍然可以使用Mac或Windows系统。开发者的生活在IDE里，pull request里，代码里。操作系统不应该是那么显眼、占空间的东西。
Justin: 我认为，有一天你会雇到一个不适合的人，在团队中无法有效工作。Donnie Berkholz，我们的产品总监，有一个很棒的演讲，他很多年前在Redmonk举办的活动上讲的关于assholes怎么毁掉你的开源项目。这点也非常适用于公司的情况，指的是那些消极的、对你的目标有破坏性的人。尽早意识到他是谁，并解决这个问题真的很重要。