Cloud battles: where will open source land?

In fact, by 2022, 75% of all databases will be deployed or migrated to a cloud platform, with only 5% ever considered for repatriation to on-premises, according to Gartner, Inc. As companies of all sizes modernize their applications and systems, they will overwhelmingly turn to the cloud to remain competitive, grow revenue and adapt to changing markets.

The intersection of cloud and open source has received a lot of attention lately. Some public cloud providers have been accused of strip-mining open source, in particular open source databases. Taking the open source software and putting it on their shelves for retail and profit while simultaneously failing to give back to the communities who built the code. Amazon in particular is driving more database revenue ($6.3b in 2019 according to Gartner) than all the open source database companies, that created the software, combined.The cloud no doubt furthers the reach of open source but what is the ultimate impact on the vendors that are inventing and developing the open source software?

This ‘David and Goliath’ battle between those who contribute to the development of open source platforms and some of the largest cloud providers in the world, will be a defining characteristic of the industry moving forward.

Open source: A paradigm shift in perception

In 2018, the totals from IPOs and acquisitions generated by open source technology was calculated at around $107b. A decade ago, this could not have been predicted, with widespread resistance from investors holding concerns over the reliability (both maturity and security) of open source technology dominating the market. Since then, the industry has undergone a shift that has seen open source truly arrive.

Open source platforms have exceeded all expectations to become a force to be reckoned with in the database space. By making a clear mark on Wall Street, open source technology has successfully demonstrated the profitability, reliability and security of its service. A number of high-profile acquisitions and IPOs has helped to cement the current status of open source.

To cite just a few of these, MongoDB went public in October 2017 and Elastic in 2018. Microsoft completed its acquisition of Github the same year, and in 2019, IBM closed its landmark acquisition of Red Hat for $34bn. 2018 is widely recognised as a breakout year for open source and since then, things have only gotten bigger.

Strip-mining: Pick on someone your own size!

Almost ten years ago, Amazon pioneered the first generation of databases on the internet. But in classic retail fashion they simply strip-mined the best of open source databases, and made it available on their shelf without an ability to innovate on the database itself. Databases are some of the most complex software products today. Wrapping open source databases with proprietary technology to deliver the AWS RDS service essentially locked out the community, and made it difficult even for Amazon to keep the core database technology up to date and evolving as quickly as the database creators are. Because of this lockout, most versions of AWS RDS and Aurora are years behind the database technology available today - Aurora is built on MySQL 5.7 - a five year old release of MySQL. Many bug fixes and security updates are missing despite being readily available. They certainly got the ball rolling in terms of making the public cloud a staple for everyone using databases, but by not partnering with the open source database creators they missed a critical opportunity.

In response, many open source database vendors adopted new licenses to prevent public clouds from strip-mining. MongoDB released the Server Side Public License in 2018, restricting its database from being used by service providers unless they were willing to share the source code of their applications used to provide the service which they are monetising. Confluent, behind the Kafka project, took a similar approach after AWS released a competitive Managed Streaming Kafka. MariaDB created the Business Source License for some of its products which requires a subscription to use the software in production and converts to an open source license after a certain period of time. The BSL has been adopted by several companies including Cockroach Labs, Sentry and Zero Tier.

Strip-mining leads to opportunity

In spite of all this, what matters most is customer experience. First generation cloud databases emphasized convenience and price, aiming to serve the lowest common denominator. For companies that expected more, they’ve experienced heartache when it comes to using off the shelf, strip-mined public cloud databases. We’ve heard of core applications that went down during business-critical moments for Fortune 500 companies, downright comical support responses or scalability and performance that do not meet enterprise requirements.

This has led to a unique opportunity to address a customer need that open source vendors can respond to. By delivering a cloud database that’s “from the source”, built by the original database creators, companies get the attention, performance and expertise they deserve. A database solution based on an enterprise-grade solution, not just the strip-mined community bits. The newest and latest version of the technology now, not years later. Freedom of multi-cloud, not locked into one. The benefit of continuous innovation – buying direct from the database vendor means those monies are invested back into open source innovation, rather than buying off the shelf where the profits do not make it back to open source development.

The way forward

Despite consistent abuse of community-driven technology by large cloud vendors, there is a bigger opportunity than ever for open source vendors to innovate by building a new generation of cloud offerings and we are already seeing a multitude of new services redefine the way we interact with databases such as MongoDB Atlas, MariaDB SkySQL and Elasticsearch Service on Elastic Cloud. The universal need for accessible yet robust database services has never been higher.

First generation cloud databases made it easy to deploy a database anywhere, yet also exposed certain gaps: stunted features that eliminate the ability to lift and shift to the cloud; limited configurations; mission-critical capabilities that are missing; entire industries alienated; and a lack of critical database expertise.

Now, the second generation cloud database is allowing customers to do so much more than before. The increasing power of database solutions to support various workloads –  transactions, analytics (or both simultaneously) – are enriching the customer experience to offer a more efficient service optimised by industry-led cloud supporting architecture.

The cloud has laid a path for open source to succeed and be stronger than ever.