The most fascinating tech company story during this ongoing COVID-19 pandemic is Zoom. It’s meteoric rise in usage, coupled by a similar surge in its stock price, is met with increasing scrutiny of its security practices, a drop in stock price, and a whole lot of FUD (Fear, Uncertainty, Doubt) and piling on.

The rollercoaster ride will continue for quite a while longer. But the reality with Zoom, like all realities, exists in between the extremes, along a spectrum, and among a basket of tradeoffs and nuances that usually get lost in this confusing world. Yet, it’s important and worthwhile to find where that middle ground is, so I’ll give it a try.

China FUD

FUD is an age-old marketing tactic. The term dates back to the early 20th century, but was popularized by its wide use in the technology industry, selling hardware and software to large enterprises. IBM was an early and prolific user. Microsoft did its fair share of it as well. It’s now commonly used by companies, big and small, looking to gain an edge in perception in front of customers in a competitive situation.

FUD used to be about technical uncertainty -- big incumbents squeezing out startups by pushing disinformation, so customers would go for or stay with the “safe choice”, while potentially losing out on new innovations and better products.

In our increasingly interconnected economy and complex international relations, FUD has now taken on a new form. Its use is no longer about technology alone, but technology shrouded in the fog of geopolitics and nationalism. And that brings us to Zoom.

There has been a lot of media attention paid to a Citizen Lab’s analysis of Zoom’s poor implementation of end-to-end encryption (E2EE). Among its findings, the encryption/decryption keys of some Zoom calls that happen in North America are routed to servers located in Beijing, and Zoom employs more than 700 engineers in China for R&D and development purposes, the latter of which is consistently disclosed as a “Risk Factor” in Zoom’s regular filings with the SEC, so it can hardly be considered a “finding”. All this information would’ve been fair game, but Citizen Lab took the unfortunate, cringe-worthy step of stoking FUD with headings like “A US Company with a Chinese Heart?”. And this sentiment has been picked up by multiple media outlets, without further analysis or context, and of course in the notorious Twitterverse.

Furthermore, Citizen Lab’s discovery of the Beijing servers did not include the fact that those servers are AWS servers. Zoom’s response blog post painstakingly noted that all its data centers in China are run by either Telstra, an Australian company, or AWS, an American company. Very little of these details made it into the subsequent media headlines. That’s sadly how FUD takes off and morphs into a narrative of its own.

To be clear, I understand that non-Chinese tech companies must form some kind of joint-ventures with local Chinese entities in order to operate. The AWS in China is not exactly the same as the AWS in the US, but it’s also not the same as a homegrown Chinese cloud provider; it exists in the middle, a nuance that is all but lost.

Concerns around Chinese espionage, both corporate and national intelligence, are no doubt legitimate and important. But generalizing every server request that happens inside China to be a cyber-spying field day is more fiction than fact, and ignores the messy complexity of how data exchange works in China -- between private companies and government, between state-owned enterprises and government, and even between different government departments. Tech companies, especially ones listed as public companies in the U.S. or Hong Kong, like Alibaba and Tencent (and Zoom), often resist giving the government user data to protect their commercial interests; they don’t just roll over like many people assume. (The Financial Times did a deep dive of this “messiness” recently that’s worth reading.)

And if employing engineers in China automatically makes your product a security threat, then we ought to place VMWare, IBM, Google, and Microsoft on this “naughty list” too, all of which employ R&D teams up to the thousands in multiple Chinese cities and make products occupying much more critical layers of the infrastructure stack than videoconferencing.

I’m not suggesting that Citizen Lab is intentionally pushing FUD to help Zoom’s competitors. But it will be used that way anyway. And I believe Citizen Lab’s overall work is very informative and valuable; its recent analysis of China’s social media censorship of all coronavirus related information was rigorous and objective. Frankly, it’s because of its good track record and reputation that made the FUD in its Zoom analysis both unfortunate and puzzling.

Pointing out the FUD by no means exonerates Zoom’s lack of security, privacy, and data collection oversight. Its false claim of having implemented E2EE is perhaps its worst mistake yet.

This mistake, however, is also the source of an important under-explored tradeoff.

Performance Tradeoff

A big reason why Zoom was a rocketship even before COVID-19 was because of its “just work” user experience, helping its product standout among much better known tools like Skype, WebEx, and Google Hangout. What endeared it to its users are also seemingly frivolous but office worker-friendly features like virtual background, beautification filters, and Snapchat-style filters. Zoom is very much a poster child of the trend of “consumerization of enterprise technology”, which I’ve discussed in the Dropbox context before.

All these features have performance characteristics and implications. There is generally at least some performance overhead when encryption is in place for other technical scenarios. It’s not a question of if, but how much. In the report by The Intercept, which initially exposed Zoom’s lack of proper E2EE implementation and dubious marketing claim, the cryptographer and Johns Hopkins computer science professor, Matthew Green pointed out:

...that group video conferencing is difficult to encrypt end to end. That’s because the service provider needs to detect who is talking to act like a switchboard, which allows it to only send a high-resolution videostream from the person who is talking at the moment, or who a user selects to the rest of the group, and to send low-resolution videostreams of other participants. This type of optimization is much easier if the service provider can see everything because it’s unencrypted.

Professor Green goes on to say:

If [Zoom is] all end-to-end encrypted, you need to add some extra mechanisms to make sure you can do that kind of ‘who’s talking’ switch, and you can do it in a way that doesn’t leak a lot of information. You have to push that logic out to the endpoints...This isn’t impossible...as demonstrated by Apple’s FaceTime, which allows group video conferencing that’s end-to-end encrypted.

This insight says to me that the fact that Zoom has been able to maintain its quality of service plus its many features, without any notable outage, while getting crushed by demand is at least in part because it did not implement E2EE by the books. (Maybe that’s also why no one is picking FaceTime as their work-from-home video conferencing tool of choice.)

Thus far, there has not been any thorough performance analysis done on Zoom with a perfectly implemented E2EE for every endpoint of every videoconference. (Readers: please hold me accountable and send me any you’ve found!) But there is a clear tension and set of tradeoffs between having end-to-end encryption and maintaining the level of user experience that Zoom has been delivering, and will have to continue delivering to stay competitive against much bigger but less performant players, like Microsoft, Google, and Cisco.

Are we letting the perfect be the enemy of the good? That’s at least a fair question to pose and think about.

What makes Zoom Zoom -- a low friction, easy to use way to communicate “face to face” -- may no longer be true if E2EE is perfectly implemented for every single type of call. These may or may not be tradeoffs worth making; it really comes down to the use case. Surely a meeting among government officials or corporate executives should have a high level of security and encryption, and these types of customers would pay for it. But is it really necessary for my hip-hop dance instructor, who is relying on hosting Zoom classes with a virtual empty studio background as his only source of income right now? Or the elementary or middle school teachers, most of whom are not highly technical users, trying to educate and keep their young students engaged while at home?

Zoom is most definitely at fault for not being honest and transparent about E2EE and already suffering the consequences, with lost customers, terrible PR, and a class action lawsuit filed by a shareholder. It also can’t rewind the clock and return to its not-so-distant past as just another B2B enterprise company; ordinary consumers around the world have found Zoom, and they won’t suddenly un-find it.

So taking a step back, there has to be a middle ground when it comes to performance tradeoff, where encryption requirements are contextualized in the use cases and corresponding user experience. Otherwise Zoom, business customers, and consumers may all be worse off.

Open Source to Maturity

There’s no question that Zoom, as a product and a company, will have to grow up and mature fast, fighting through both real technical challenges and plenty of FUD that’s already out there.

One path that may accelerate that growth and maturity is to open source its codebase.

From a reputational angle, open sourcing its codebase is the most effective way to establish trust and transparency. You can claim to have removed the China-based servers from the whitelist and the mistake will never happen again, but letting interested parties see and test the code and routing is much more convincing. Sunlight is always the best disinfectant.

From a technical level, open source has proven to be the most effective way to develop robust and secure software. Linux is the obvious example. (It was also the subject of much FUD from Microsoft years ago.)

In this Forbes profile of Eric Yuan, Zoom’s founder and CEO, published on April 3 and written while all this sequence of events was unfolding, Yuan hinted that he would consider open sourcing Zoom “in the next several years”. Fast forward to April 8, during Yuan’s first weekly “Ask Eric Anything” webinar on Zoom security, he now plans to open source the customer-side key-generation portion of Zoom’s encryption mechanism as it's being developed fairly soon, which would be a welcoming step if it happens.

Meanwhile another open source alternative to Zoom, Jitsi, is picking up steam, as both a standalone solution and the core to 8x8’s videoconferencing product, which has experienced a 50-times usage growth during COVID-19.

Of course, to open source something is not as simple as just tossing a codebase onto a public GitHub repository. You need proper documentation, clear processes for developers to contribute or file issues and bugs, as well as proper licensing. It’s real work; a topic I’ve worked on and written about a lot previously.

Nevertheless, the sooner Zoom can get ready to open source important components of its codebase, the better its product, technology, and reputation will be.

For what it’s worth, Zoom has demonstrated its ability to act quickly to: fix and change its product, take on new measures, and apologize over and over (and over) for its mistakes without appearing defensive. Crisis communication is never easy. I’ve been in similar shoes in the government and political context. I don’t envy anyone working in Zoom’s communications and engineering department right now.

Yuan has shown solid leadership and the needed vision, steadiness, and decisiveness to convince ex-Facebook Chief Security Officer, Alex Stamos, to help out, which is an important step in the right direction. Late last year, Yuan said during an enterprise technology event hosted by GGV Capital that taking a company public is not much more than high school graduation -- there’s a long road ahead. He grasps the fact that his creation is still young, immature, and untested in many ways. Unfortunately, Zoom won’t get four years worth of college time to meet those challenges.

If you like what you've read, please SUBSCRIBE to the Interconnected email list. New posts will be delivered to your inbox (twice per week). Follow and interact with me on: Twitter, LinkedIn.

为Zoom说句话：中国FUD，性能权衡，开源未来

在这场新冠疫情的大势中，一个最吸引人的科技公司故事就是Zoom。从使用率的飞速上涨，到股价飙升，到大家对其安全措施的漏洞开始严格注意，到股价下跌，还有在舆论中大量的FUD（恐惧、不确定性、怀疑；Fear, Uncertainty, Doubt）。

一波三折，像坐过山车一样，而且还会持续相当一段时间。但是Zoom的现实，和所有的现实一样，存在于极端之间，且常在一个混乱的世界里的各种权衡中迷失。但是，找到一个客观的中间点是很重要、很值得的，所以我想在这篇文章里尝试一下。

中国FUD

FUD是一种由来已久的营销策略。这一术语可以追溯到20世纪初，但普遍是在科技行业中，在向大型企业销售硬件和软件时广泛使用。IBM是个“老用户”了，微软也做了不少。现在大大小小的科技公司都会多多少少在客户面前，在与竞争对手一争高低时用点FUD来取得优势。

在使用FUD时，通常会从技术层面上戳对手的短处和不稳定性：大企业会通过虚假信息来排挤创业公司，客户可能被误导从而做一个保守的“安全选择”，也因此常常会失去使用经过创新的、更好的产品的机会。

在我们日益紧密联系的国际化经济和复杂的国际关系中，FUD演变出了新的形式。它的“内容来源”已不再仅仅是技术，还加上了地域政治关系和民族主义色彩。

最近，一篇Citizen Lab（多伦多大学的一个学术小组）对Zoom端到端加密（E2EE）落实程度不够好的分析引起了媒体的广泛关注。调查中的主要发现是：一些源于北美的Zoom视频会议的加密/解密密钥被路由到位于北京的服务器，而且Zoom在中国雇佣了700多名工程师做研发工作（后者一直在Zoom向美国证券交易委员会提交的常规文件中公开披露过多次，类为“风险因素”，所以也算不上是什么“发现”）。所有这些信息本身都没什么问题，可以好好讨论，但Citizen Lab采取一种故意渲染的方式，用“一家有中国心的美国公司”这种耸动的标题煽动对Zoom的FUD，已经被多家媒体报道直接接受，而不加任何分析和解释，当然也包括推特。

此外，Citizen Lab对其发现的北京服务器并没有包括所有的信息，比如这些服务器都是AWS运营的服务器这一事实。Zoom在回复以上报道的博客文章中煞费苦心地指出，它在中国所有的数据中心要么由澳大利亚的Telstra公司运营，要么由美国的AWS运营。但这些细节很少登上随后的媒体头条。FUD就这样脱胎换骨，演变成了自己的故事。

我明白，非中国科技企业必须与中国本土企业建立某种合资企业才能运作。中国的AWS与美国的AWS并不完全相同，但它也与中国本土的云厂商有所差别；它在两者中间，这种细微差别在舆论中几乎消失殆尽。

针对中国间谍活动的担忧，无论是商业还是国家机密，都是合理的。但将中国境内的每一台服务器都一刀切地概括为收集情报的想法却是幻想多于事实，也忽略了中国里数据的来来往往的混乱程度与复杂性。这里面的复杂关系有多种：私营企业与政府、国有企业与政府，甚至包括不同政府部门之间。科技公司，尤其是在美国或香港上市的公司，比如阿里巴巴和腾讯（和Zoom），常常拒绝上交给政府用户的数据，因为这种做法会威胁它们的商业利益；这层关系并不像许多人默认的那样，政府要公司交出用户信息，它们就会乖乖上交。（英国《金融时报》最近对这些复杂关系进行了深度剖析，值得一读。）

如果在中国雇佣工程师会自动把你的产品变成一种安全威胁，那么我们也应该公平地把VMWare、IBM、Google和Microsoft列入这个“黑名单”。所有这些公司都在中国多个城市雇佣了高达数千人的研发团队，开发的产品在基础设施层面里远远比视频会议应用要核心、重要得多。

我并不是说Citizen Lab有意推动FUD来帮助Zoom的竞争对手。但渲染的方式肯定会被Zoom对手这么用。我其实一直觉得Citizen Lab的整体工作是信息量很高，很有价值的。它最近对中国社交媒体审查删除有关新冠病毒负面信息的分析是很严谨和客观的。正是因为它良好的业绩和声誉，才使得其对Zoom分析中的FUD既不幸又令人费解。

指出这些FUD并不证明就可以免除Zoom在安全、隐私和数据收集监管方面的失责。它错误地宣称产品已经实现了E2EE但实际上并没有，可能是它迄今为止犯过的最严重的错误。

然而，这一错误也是一个重要的未充分探讨的产品性能权衡的问题。

性能权衡

即使在疫情爆发之前，Zoom以火箭般的速度快速增长，其中一个重要原因是其产品极佳的用户体验，帮助它在多种竞品中脱颖而出，比如Skype、WebEx和Google Hangout等。使它受到用户喜爱的还有看似轻浮但对上班族很贴心的功能，比如虚拟背景、美颜和Snapchat风格的视频滤镜。Zoom是“企业产品人性化”趋势的典型代表，我在之前谈论Dropbox的文章中讨论过这个趋势。

所有这些功能都有不同的性能特征和影响。在大部分技术场景下进行加密，通常都会对性能有一些的影响。这不是是否的问题，而是大小的问题。在最初暴露Zoom缺乏适当的E2EE和有误导嫌疑的市场宣传的媒体报道中，密码学家及Johns Hopkins大学计算机学教授Matthew Green指出：

…群组视频会议很难做到端到端加密。这是因为提供服务的厂商需要检测谁在说话，就像一个交换机，这样它才可以只从正在说话的人那里发送一个高分辨率的视频流，或者让用户选择谁发送给组中的其他人，并给其他不说话的参与者发送低分辨率视频流。如果提供服务的厂商可以看到所有内容，因为它是未加密的，那么落实这种优化就容易的多。

Green教授接着说：

如果【Zoom】所有的视频会议都是端到端加密的，你需要添加一些额外的机制，以确保你可以做那种‘谁在说话’的切换，而且你可以用一种不会泄露很多信息的方式来完成。你必须把业务逻辑推到端点……这并非不可能……苹果的FaceTime就落实了这一点，FaceTime允许端到端加密的群组视频会议。

这位专家的看法告诉我，Zoom之所以能在疫情爆发，使用率暴涨的情况下没有出现任何明显的服务中断，同时还能保持产品服务的质量和许多功能，至少在一定程度上是因为它没有按照书本上的标准实现E2EE。（也许这也是为什么没有人选择FaceTime作为他们在家办公的视频会议工具。）

到目前为止，我还没有看到如果Zoom在每个视频会议的每个端点上都完美实现E2EE后对整个产品性能影响的分析。（读者：如果你们看到类似信息，麻烦一定要发给我！）但是可以肯定的是，端到端加密和Zoom一贯提供的用户体验水平之间是个权衡的关系。“用户体验”也是Zoom要想继续和微软、谷歌，思科等巨头们继续竞争，而不可妥协的优势。

我们对完美加密的追求会不会导致丧失一个出色的产品？这起码是个应该提出和值得思考的问题。

如果E2EE完美地落实了，那也许当初大家喜爱的Zoom，一个低摩擦、易用的“面对面”通信方式，可能就不如从前了。这个妥协可能是值得做的，也可能是不值得做的；这纯粹取决于客户用例。政府官员或企业高管之间的会议当然应该具有高度的安全性和加密性，这些类型的客户也有能力为安全功能付费。但对我的嘻哈舞蹈老师来说，真的有必要吗？对中小学老师来说，大部分都不太懂技术，为了继续教育因为疫情而锁在家里的小朋友们，真的有必要吗？

Zoom因为对E2EE问题的不诚实和不透明，已经开始承受后果了，比如丧失客户、负面公关效应和一个股东引发的群体诉讼，这无疑是Zoom的错。它再也回不到过去，只做一家默默赚钱的B2B公司了；全世界的网民都已经听说了Zoom，他们不会突然失忆的。

所以退一步看，整个性能权衡的问题必须有一个中间点，加密的同时也要考虑用例和相应的用户体验。否则Zoom、企业客户和广大散户都会失去自己在乎的东西。

借开源来成长

毫无疑问，无论是作为一个产品和还是一家公司，Zoom必须迅速成长和成熟起来，并且要面对的巨大的技术挑战和大量的舆论FUD。

一条也许会加速这种成长和成熟的途径就是开源代码。

从提高声誉和信誉角度来看，开源代码是建立信任和透明度最有效的途径。公司可以自己声明已经从白名单中删除了基于中国的服务器，也不允许同样错误再发生，但远远没有开源代码让所有感兴趣的人自己看，自己做测试有说服力。阳光总是最好的消毒剂。

从技术层面来看，开源已经被证明是开发既结实又安全的软件的最好方式。Linux就是一个最好的例子。（若干年前，Linux也是源于微软的FUD的受害者。）

在4月3日《福布斯》出版的对Zoom创始人兼首席执行官袁征的长篇介绍中，他暗示了将考虑“在未来几年”把Zoom开源。快进到4月8日，在袁征举办的第一个关于Zoom安全问题的每周网络问答会上，他说自己已经计划在近期开源Zoom在客户端加密机制及密钥生成部分的代码。这事如果发生，是很值得庆幸的一步。

与此同时，Zoom的一个开源替代品Jitsi已经在兴起，它既是一个独立的视频会议解决方案，也是8x8视频会议产品的核心。在新冠疫情期间，8x8的使用量增长了50倍。

当然，开源并不是简单地将一个代码库放到GitHub上就完了。这个过程需要好的文档、清晰的代码贡献和报告问题和bug的流程，以及得当的他人使用授权许可。这都是在开源前必须做的工作。这个话题我个人亲身体会过，也就它写过很多文章。

尽管如此，Zoom越早准备好开源代码库的重要组件，它的产品、技术和声誉就会变得越好。

Zoom团队已经证明自己快速行动的能力，从修复和改进产品，到采取新的措施，并为自己的错误一次又一次（再又一次）不厌其烦地道歉。做好危机公关绝非易事。我在政府和政治领域有过类似的经历。我一点都不羡慕现在在Zoom做公关传媒和技术部门工作的人。

袁征展示了他在危机下的领导能力和必要的远见、坚定和果断，并且说服了前Facebook首席安全官Alex Stamos来援助他的团队，这一步迈的非常正确，也非常重要。去年年底，袁征曾在GGV资本举办的一次企业技术活动上表示，上市好比高中毕业，未来还有很长的路要走。他早已意识到自己的创作还很年轻，还不成熟，在很多方面还没有经过历练。可惜的是，Zoom不会有四年上大学的时间来应对这些挑战。

如果您喜欢所读的内容，请用email订阅加入“互联”。每周两次，新的文章将会直接送达您的邮箱。请在Twitter、LinkedIn上给个follow，与我交流互动！