Open Source in China: Next Four Years

Last May, I wrote a three-part series that surveyed the entire open source landscape in China:

• Part I: the Players
• Part II: the Game
• Part III: the Trends

Since then, the landscape has changed significantly as “open source” continues to receive top-down attention from the central government. Recently, the Ministry of Industry and Information Technology (MIIT, China’s main regulator of its tech industry) released a new set of development planning guidance. “Open source” is prominently featured; the term was used 27 times.

This guidance specifically called out the establishment of China’s first home-grown open-source foundation -- the OpenAtom Foundation -- as a major accomplishment (for more on the OpenAtom Foundation, see my previous analysis). Looking ahead, the guidance explicitly sets a target of building 2-3 open source communities with “global influence” by 2025.

The MIIT guidance is focused on advancing and modernizing China’s industrial technology infrastructure and supply chain. The infrastructure layer is where open source software tends to excel starting with Linux, the open source operating system that runs most of the world’s servers. China’s open source community has been organically growing since the early 2000s. The country now sports the world’s second largest developer community, with a burgeoning if not crowded open source ecosystem. Many projects have 5-figure GitHub stars, thousands of forks, and hundreds of contributors, but those are mostly vanity metrics. Only projects with strategic technical value have the potential to reach a global developer audience.

With the MIIT expressing both top-level approval and high expectation of open source, which “Made in China” open source projects have the best chance of emerging as both a “local champion” and a “global influencer” by 2025? I share my prediction of six leading candidates -- three Big Tech projects, three independent projects -- and why they could become global projects.

Big Tech Projects

The three Big Tech projects are Baidu’s Apollo (self-driving), Huawei’s OpenHarmony (operating system), and Alibaba’s OpenXuantie (RISC-V based semiconductor design).

Apollo: First launched in 2017, Apollo has emerged as one of the leading self-driving open source projects. It’s at the heart of Baidu’s Apollo Go robotaxi service, which “beta launched” in a neighborhood in Beijing. That neighborhood just happens to be one of the venues of the upcoming Winter Olympics. Even though this Winter Olympics is receiving lots of geopolitical pushback from Western countries -- the US, Canada, Australia, and the UK have all announced diplomatic boycotts of the competition -- it will still be a major product launch moment for China’s self-driving ambition in front of a global audience.

How well Apollo works during the Olympics will indicate whether Baidu’s goal of launching the Apollo Go robotaxi service in 65 cities by 2025 is realistic or not. But Baidu’s goal aside, it is likely that Apollo as an open source project with a permissive license (Apache 2.0) has already been copied, reused, and modified to run other self-driving vehicles made by other manufacturers. By 2025, Apollo may well emerge as the leading open source self-driving alternative to Tesla’s self-driving software stack, which is at the moment totally closed source and proprietary. At first glance, Apollo’s documentation and ReadMe on GitHub are all in English and look well-structured and well-written. This is a positive signal that the project team is investing in making Apollo accessible to non-Chinese developers.

If Tesla’s system is the iOS of self-driving, Baidu’s Apollo may become self-driving’s Android.

Huawei’s OpenHarmony: first started in 2012, Huawei’s development of OpenHarmony intensified in 2019 to reduce its reliance on the Android operating system due to US sanctions. The OpenHarmony project also became the “anchor project” of the OpenAtom Foundation, which Huawei is heavily involved in.

While OpenHarmony’s development was mostly a reaction to possible future US sanctions, there is a proactive, strategic element to OpenHarmony potentially becoming an alternative OS to both iOS and Android. For countries like Russia that are also concerned about US sanctions, OpenHarmony could be an appealing option.

Even though the MIIT guidance explicitly mentioned OpenHarmony (and no other project), the project is still relatively young and hosted on Gitee, China’s domestic alternative to GitHub with limited appeal to non-Chinese developers. From a strategic technical perspective, OpenHarmony has global influence potential, but the team has a long way to go to appeal to the global developer community.

Alibaba’s OpenXuantie: this project is the youngest of the three Big Tech projects, so it’s my “dark horse” pick. It was just open sourced in October, so Xuantie has hardly any community presence or traction, whether in Chinese or English.

Why do I think OpenXuantie has the promise to reach the kind of “global influence” that MIIT is targeting by 2025?

First, it is incubated within Alibaba, which has the strongest open source technical and community talent among all of China’s tech giants. Second, it leverages RISC-V, the open source semiconductor design architecture. (For more on RISC-V, please see my previous writings.) As far as I know, Alibaba still sports the fastest RISC-V based processor. Now it is taking a profound step forward to open source its semiconductor design development via OpenXuantie.

Given semiconductor’s obvious strategic importance to China, as well as countries like India, Pakistan, Japan, Korea, and many European countries, open source projects that can take RISC-V to the next level will surely gain global influence. Of course, the future of the semiconductor industry is anything but certain, with the US Federal Trade Commission challenging Nvidia’s acquisition of Arm on antitrust grounds. But RISC-V will be a force no matter what happens to the Nvidia-Arm deal, and OpenXuantie may play a big role in RISC-V’s continuous rise.

Independent Projects

The three independent projects are TiDB / TiKV (distributed database), OpenResty (API gateway), and OceanBase (distributed database).

TiDB / TiKV: started in 2015, TiDB is one of the most active open source distributed databases that is not incubated by any tech giant. TiKV is the key-value storage layer of the database, which was donated to the Cloud Native Computing Foundation in 2018 as a separate project. Thus, this layer technically belongs to the Foundation, not PingCAP, the startup that originally created both TiDB and TiKV in tandem.

Disclaimer: I worked for two years at PingCAP to launch its global expansion operation. I don’t have any knowledge of the company’s latest planning or roadmap. This analysis is purely based on my outside observation of these open source projects.

Because databases are a crucial layer of any infrastructure stack, TiDB / TiKV very much fits the MIIT’s strategic focus on China’s industrial technology infrastructure and supply chain. And because of the projects’ popularity among database engineers, many of whom are open source enthusiasts, the core technology has advanced quickly. In fact, TiDB / TiKV have been considered as a good enough home-grown alternative to the likes of Oracle and IBM since 2018. With users like Square and Shopee, you can argue that TiDB / TiKV already has some global influence.

OpenResty: started in 2011, OpenResty is one of the oldest and most widely used open source projects that originated from China. This project is likely how many Western developers first came in contact with “Made in China” open source technology.

Similar to the database layer, the API gateway layer is also crucial to all flavors of infrastructure technology, especially ones that are cloud-native. In simple terms, an API gateway provides load balancing and routing of API calls and web traffic dynamically, so your system does not crash when too many requests come in, say on an online shopping holiday like Singles Day. It’s no surprise then that OpenResty was first created inside Yahoo China, then followed its creator to Taobao, then to Cloudflare. The project is now stewarded by its own foundation and commercialized by a startup, also called OpenResty.

Given its long history and wide adoption, including users like Target and Lyft, OpenResty also already has “global influence”. It will be interesting to see how big the project becomes by 2025.

OceanBase: it may be a bit of a stretch to call OceanBase an independent open source project, because it was neither independent nor open source until very recently. First incubated inside Ant Financial as the primary transactional database for AliPay, Taobao, then many other products in the vast Alibaba ecosystem, OceanBase was spun out in mid-2020 as an independent company. However, the company is still mostly owned by Ant. OceanBase’s open source history is even shorter than its independent corporate history; its core codebase was open sourced in mid-2021, only a few months before my writing this post.

However, OceanBase deserves a mention because it is literally the fastest database in the world, at least by the measurement of a well-known industry benchmark called TPC-C. I won’t go into the technical details of what this benchmark measures. What’s important to note is that not only was OceanBase first place based on the TPC-C benchmark measurement, it was twice as fast as the second place Oracle product.

Given its short open source history, it remains to be seen how much “global influence” OceanBase can accrue over the next four years. Its core technology is unquestionably top-notch. But technology alone is by no means sufficient for any open source project to reach global prominence.

Every one of these six projects is worthy of its own deep dive analysis; this overview post barely scratches the surface of their respective technology and strategic value. While we won’t know for four more years which open source projects will receive the MIIT’s “global influence” badge of honor, open source itself is clearly a force that the Chinese government has both recognized and (mostly) embraced. Thus, understanding how open source works is table stakes for anyone working in technology, foreign policy, and especially the intersection of both.