The Cost of Cloud, a Trillion Dollar Paradox

by Sarah Wang and Martin Casado

- cloud computing

- enterprise & SaaS

- networking

- growth (late stage venture)

- metrics

- cloud infrastructure

- trends 2021

- Facebook

- LinkedIn

- Twitter

There is no doubt that the cloud is one of the most significant platform shifts in the history of computing. Not only has cloud already impacted hundreds of billions of dollars of IT spend, it’s still in early innings and growing rapidly on a base of over $100B of annual public cloud spend. This shift is driven by an incredibly powerful value proposition — infrastructure available immediately, at exactly the scale needed by the business — driving efficiencies both in operations and economics. The cloud also helps cultivate innovation as company resources are freed up to focus on new products and growth.

source: Synergy Research Group

However, as industry experience with the cloud matures — and we see a more complete picture of cloud lifecycle on a company’s economics — it’s becoming evident that while cloud clearly delivers on its promise early on in a company’s journey, the pressure it puts on margins can start to outweigh the benefits, as a company scales and growth slows. Because this shift happens later in a company’s life, it is difficult to reverse as it’s a result of years of development focused on new features, and not infrastructure optimization. Hence a rewrite or the significant restructuring needed to dramatically improve efficiency can take years, and is often considered a non-starter.

Now, there is a growing awareness of the long-term cost implications of cloud. As the cost of cloud starts to contribute significantly to the total cost of revenue (COR) or cost of goods sold (COGS), some companies have taken the dramatic step of “repatriating” the majority of workloads (as in the example of Dropbox) or in other cases adopting a hybrid approach (as with CrowdStrike and Zscaler). Those who have done this have reported significant cost savings: In 2017, Dropbox detailed in its S-1 a whopping $75M in cumulative savings over the two years prior to IPO due to their infrastructure optimization overhaul, the majority of which entailed repatriating workloads from public cloud.

Yet most companies find it hard to justify moving workloads off the cloud given the sheer magnitude of such efforts, and quite frankly the dominant, somewhat singular, industry narrative that “cloud is great”. (It is, but we need to consider the broader impact, too.) Because when evaluated relative to the scale of potentially lost market capitalization — which we present in this post — the calculus changes. As growth (often) slows with scale, near term efficiency becomes an increasingly key determinant of value in public markets. The excess cost of cloud weighs heavily on market cap by driving lower profit margins.

The point of this post isn’t to argue for repatriation, though; that’s an incredibly complex decision with broad implications that vary company by company. Rather, we take an initial step in understanding just how much market cap is being suppressed by the cloud, so we can help inform the decision-making framework on managing infrastructure as companies scale.

To frame the discussion: We estimate the recaptured savings in the extreme case of full repatriation, and use public data to pencil out the impact on share price. We show (using relatively conservative assumptions!) that across 50 of the top public software companies currently utilizing cloud infrastructure, an estimated $100B of market value is being lost among them due to cloud impact on margins — relative to running the infrastructure themselves. And while we focus on software companies in our analysis, the impact of the cloud is by no means limited to software. Extending this analysis to the broader universe of scale public companies that stands to benefit from related savings, we estimate that the total impact is potentially greater than $500B.

Our analysis highlights how much value can be gained through cloud optimization — whether through system design and implementation, re-architecture, third-party cloud efficiency solutions, or moving workloads to special purpose hardware. This is a very counterintuitive assumption in the industry given prevailing narratives around cloud vs. on-prem. However, it’s clear that when you factor in the impact to market cap in addition to near term savings, scaling companies can justify nearly any level of work that will help keep cloud costs low.

Unit economics of cloud repatriation: The case of Dropbox, and beyond

To dimensionalize the cost of cloud, and understand the magnitude of potential savings from optimization, let’s start with a more extreme case of large scale cloud repatriation: Dropbox. When the company embarked on its infrastructure optimization initiative in 2016, they saved nearly $75M over two years by shifting the majority of their workloads from public cloud to “lower cost, custom-built infrastructure in co-location facilities” directly leased and operated by Dropbox. Dropbox gross margins increased from 33% to 67%  from 2015 to 2017, which they noted was “primarily due to our Infrastructure Optimization and an… increase in our revenue during the period.”

source: Dropbox S-1 filed February 2018

But that’s just Dropbox. So to help generalize the potential savings from cloud repatriation to a broader set of companies, Thomas Dullien, former Google engineer and co-founder of cloud computing optimization company Optimyze, estimates that repatriating $100M of annual public cloud spend can translate to roughly less than half that amount in all-in annual total cost of ownership (TCO) — from server racks, real estate, and cooling to network and engineering costs.

The exact savings obviously varies company, but several experts we spoke to converged on this “formula”: Repatriation results in one-third to one-half the cost of running equivalent workloads in the cloud. Furthermore, a director of engineering at a large consumer internet company found that public cloud list prices can be 10 to 12x the cost of running one’s own data centers. Discounts driven by use-commitments and volume are common in the industry, and can bring this multiple down to single digits, since cloud compute typically drops by ~30-50% with committed use. But AWS still operates at a roughly 30% blended operating margin net of these discounts and an aggressive R&D budget — implying that potential company savings due to repatriation are larger. The performance lift from managing one’s own hardware may drive even further gains.

Across all our conversations with diverse practitioners, the pattern has been remarkably consistent: If you’re operating at scale, the cost of cloud can at least double your infrastructure bill.

The true cost of cloud

When you consider the sheer magnitude of cloud spend as a percentage of the total cost of revenue (COR), 50% savings from cloud repatriation is particularly meaningful. Based on benchmarking public software companies (those that disclose their committed cloud infrastructure spend), we found that contractually committed spend averaged 50% of COR.

Actual spend as a percentage of COR is typically even higher than committed spend: A billion dollar private software company told us that their public cloud spend amounted to 81% of COR, and that “cloud spend ranging from 75 to 80% of cost of revenue was common among software companies”. Dullien observed (from his time at both industry leader Google and now Optimyze) that companies are often conservative when sizing cloud commit size, due to fears of being overcommitted on spend, so they commit to only their baseline loads. So, as a rule of thumb, committed spend is often typically ~20% lower than actual spend… elasticity cuts both ways. Some companies we spoke with reported that they exceeded their committed cloud spend forecast by at least 2X.

If we extrapolate these benchmarks across the broader universe of software companies that utilize some public cloud for infrastructure, our back-of-the-envelope estimate is that the cloud bill reaches $8B in aggregate for 50 of the top publicly traded software companies (that reveal some degree of cloud spend in their annual filings). While some of these companies take a hybrid approach — public cloud and on-premise (which means cloud spend may be a lower percentage of COR relative to our benchmarks) — our analysis balances this, by assuming that committed spend equals actual spend across the board. Drawing from our conversations with experts, we assume that cloud repatriation drives a 50% reduction in cloud spend, resulting in total savings of $4B in recovered profit. For the broader universe of scale public software and consumer internet companies utilizing cloud infrastructure, this number is likely much higher.

source: company S-1 and 10K filings; a16z analysis

While $4B of estimated net savings is staggering on its own, this number becomes even more eye-opening when translated to unlocked market capitalization. Since all companies are conceptually valued as the present value of their future cash flows, realizing these aggregate annual net savings results in market capitalization creation well over that $4B.

How much more? One rough proxy is to look at how the public markets value additional gross profit dollars: High-growth software companies that are still burning cash are often valued on gross profit multiples, which reflects assumptions about the company’s long term growth and profitable margin structure. (Commonly referenced revenue multiples also reflect a company’s long term profit margin, which is why they tend to increase for higher gross margin businesses even on a growth rate-adjusted basis). Both capitalization multiples, however, serve as a heuristic for estimating the market discounting of a company’s future cash flows.

Among the set of 50 public software companies we analyzed, the average total enterprise value to 2021E gross profit multiple (based on CapIQ at time of publishing) is 24-25X. In other words: For every dollar of gross profit saved, market caps rise on average 24-25X times the net cost savings from cloud repatriation. (Assumes savings are expressed net of depreciation costs incurred from incremental CapEx if relevant).

This means an additional $4B of gross profit can be estimated to yield an additional $100B of market capitalization among these 50 companies alone. Moreover, since using a gross profit multiple (vs. a free cash flow multiple) assumes that incremental gross profit dollars are also associated with certain incremental operating expenditures, this approach may underestimate the impact to market capitalization from the $4B of annual net savings.

For a given company, the impact may be even higher depending on its specific valuation. To illustrate this phenomenon [please note this is not investment advice, see full disclosures below and at https://a16z.com/disclosures/], take the example of infrastructure monitoring as a service company Datadog. The company traded at close to 40X 2021 estimated gross profit at time of publishing, and disclosed an aggregate $225M 3-year commitment to AWS in their S-1. If we annualize committed spend to $75M of annual AWS costs — and assume 50% or $37.5M of this may be recovered via cloud repatriation — this translates to roughly $1.5B of market capitalization for the company on committed spend reductions alone!

While back-of-the-envelope analyses like these are never perfect, the directional findings are clear: market capitalizations of scale public software companies are weighed down by cloud costs, and by hundreds of billions of dollars. If we expand to the broader universe of enterprise software and consumer internet companies, this number is likely over $500B — assuming 50% of overall cloud spend is consumed by scale technology companies that stand to benefit from cloud repatriation.

For business leaders, industry analysts, and builders, it’s simply too expensive to ignore the impact on market cap when making both long-term and even near-term infrastructure decisions.

source: CapIQ as of May 2021; note: charts herein are for informational purposes only and should not be relied upon when making any investment decision

The paradox of cloud

Where do we go from here? On one hand, it is a major decision to start moving workloads off of the cloud. For those who have not planned in advance, the necessary rewriting seems SO impractical as to be impossible; any such undertaking requires a strong infrastructure team that may not be in place. And all of this requires building expertise beyond one’s core, which is not only distracting, but can itself detract from growth. Even at scale, the cloud retains many of its benefits — such as on-demand capacity, and hordes of existing services to support new projects and new geographies.

But on the other hand, we have the phenomenon we’ve outlined in this post, where the cost of cloud “takes over” at some point, locking up hundreds of billions of market cap that are now stuck in this paradox: You’re crazy if you don’t start in the cloud; you’re crazy if you stay on it.

So what can companies do to free themselves from this paradox? As mentioned, we’re not making a case for repatriation one way or the other; rather, we’re pointing out that infrastructure spend should be a first-class metric. What do we mean by this? That companies need to optimize early, often, and, sometimes, also outside the cloud. When you’re building a company at scale, there’s little room for religious dogma.

While there’s much more to say on the mindset shifts and best practices here — especially as the full picture has only more recently emerged — here are a few considerations that may help companies grapple with the ballooning cost of cloud.

Cloud spend as a KPI. Part of making infrastructure a first-class metric is making sure it is a key performance indicator for the business. Take for example Spotify’s Cost Insights, a homegrown tool that tracks cloud spend. By tracking cloud spend, the company enables engineers, and not just finance teams, to take ownership of cloud spend. Ben Schaechter, formerly at Digital Ocean, now co-founder and CEO of Vantage, observed that not only have they been seeing companies across the industry look at cloud cost metrics alongside core performance and reliability metrics earlier in the lifecycle of their business, but also that “Developers who have been burned by surprise cloud bills are becoming more savvy and expect more rigor with their team’s approach to cloud spend.”

Incentivize the right behaviors. Empowering engineers with data from first-class KPIs for infrastructure takes care of awareness, but doesn’t take care of incentives to change the way things are done. A prominent industry CTO told us that at one of his companies, they put in short-term incentives like those used in sales (SPIFFs), so that any engineer who saved a certain amount of cloud spend by optimizing or shutting down workloads received a spot bonus (which still had a high company ROI since the savings were recurring). He added that this approach — basically, “tie the pain directly to the folks who can fix the problem” — actually cost them less, because it paid off 10% of the entire organization, and brought down overall spend by $3M in just six months. Notably, the company CFO was key to endorsing this non-traditional model.

Optimization, optimization, optimization. When evaluating the value of any business, one of the most important factors is the cost of goods sold or COGS — and for every dollar that a business makes, how many dollars does it cost to deliver? Customer data platform company Segment recently shared how they reduced infrastructure costs by 30% (while simultaneously increasing traffic volume by 25% over the same period) through incremental optimization of their infrastructure decisions. There are a number of third-party optimization tools that can provide quick gains to existing systems, ranging anywhere from 10-40% in our experience observing this space.

Think about repatriation up front. Just because the cloud paradox exists — where cloud is cheaper and better early on and more costly later in a company’s evolution — exists, doesn’t mean a company has to passively accept it without planning for it. Make sure your system architects are aware of the potential for repatriation early on, because by the time cloud costs start to catch up to or even outpace revenue growth, it’s too late. Even modest or more modular architectural investment early on — including architecting to be able to move workloads to the optimal location and not get locked in — reduces the work needed to repatriate workloads in the future. The popularity of Kubernetes and the containerization of software, which makes workloads more portable, was in part a reaction to companies not wanting to be locked into a specific cloud.

Incrementally repatriate. There’s also no reason that repatriation (if that’s indeed the right move for your business), can’t be done incrementally, and in a hybrid fashion. We need more nuance here beyond either/or discussions: for example, repatriation likely only makes sense for a subset of the most resource-intensive workloads. It doesn’t have to be all or nothing! In fact, of the many companies we spoke with, even the most aggressive take-back-their-workloads ones still retained 10 to 30% or more in the cloud.

While these recommendations are focused on SaaS companies, there are also other things one can do; for instance, if you’re an infrastructure vendor, you may want to consider options for passing through costs — like using the customer’s cloud credits — so that the cost stays off your books. The entire ecosystem needs to be thinking about the cost of cloud.

*     *     *

How the industry got here is easy to understand: The cloud is the perfect platform to optimize for innovation, agility, and growth. And in an industry fueled by private capital, margins are often a secondary concern. That’s why new projects tend to start in the cloud, as companies prioritize velocity of feature development over efficiency.

But now, we know. The long term implications have been less well understood — which is ironic given that over 60% of companies cite cost savings as the very reason to move to the cloud in the first place! For a new startup or a new project, the cloud is the obvious choice. And it is certainly worth paying even a moderate “flexibility tax” for the nimbleness the cloud provides.

The problem is, for large companies — including startups as they reach scale — that tax equates to hundreds of billions of dollars of equity value in many cases… and is levied well after the companies have already, deeply committed themselves to the cloud (and are often too entrenched to extricate themselves). Interestingly, one of the most commonly cited reasons to move the cloud early on — a large up-front capital outlay (CapEx) — is no longer required for repatriation. Over the last few years, alternatives to public cloud infrastructures have evolved significantly and can be built, deployed, and managed entirely via operating expenses (OpEx) instead of capital expenditures.

Note too that as large as some of the numbers we shared here seem, we were actually conservative in our assumptions. Actual spend is often higher than committed, and we didn’t account for overages-based elastic pricing. The actual drag on industry-wide market caps is likely far higher than penciled.

Will the 30% margins currently enjoyed by cloud providers eventually winnow through competition and change the magnitude of the problem? Unlikely, given that the majority of cloud spend is currently directed toward an oligopoly of three companies. And here’s a bit of dramatic irony: Part of the reason Amazon, Google, and Microsoft — representing a combined ~5 trillion dollar market cap — are all buffeted from the competition, is that they have high profit margins driven in part by running their own infrastructure, enabling ever greater reinvestment into product and talent while buoying their own share prices.

And so, with hundreds of billions of dollars in the balance, this paradox will likely resolve one way or the other: either the public clouds will start to give up margin, or, they’ll start to give up workloads. Whatever the scenario, perhaps the largest opportunity in infrastructure right now is sitting somewhere between cloud hardware and the unoptimized code running on it.

Acknowledgements: We’d like to thank everyone who spoke with us for this article (including those named above), sharing their insights from the frontlines.

Companies selected denoted some degree of public cloud infrastructure utilization in 10Ks

The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation. In addition, this content may include third-party advertisements; a16z has not reviewed such advertisements and does not endorse any advertising content contained therein.

This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at https://a16z.com/investments/.

Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.

Opinion

Opinion: Andreessen Horowitz is dead wrong about cloud

Rich Hoyer

June 10, 2021 1:01 PM

Image Credit: MR.Cole_Photographer/Getty Images

How open API standards will transform financial services

Open standards will have a huge impact on driving innovation in banking. Learn the status in the U.S. – and the bold new opportunities open standards are set to usher in.

Register here

The Transform Technology Summits start October 13th with Low-Code/No Code: Enabling Enterprise Agility. Register now!

In The Cost of Cloud, a Trillion-Dollar Paradox, Andreessen Horowitz Capital Management’s Sarah Wang and Martin Casado highlighted the case of Dropbox closing down its public cloud deployments and returning to the datacenter. Wang and Casado extrapolated the fact that Dropbox and other enterprises realized savings of 50% or more by bailing on some or all of their cloud deployments in the wider cloud-consuming ecosphere. Wang and Casado’s conclusion? Public cloud is more than doubling infrastructure costs for most enterprises relative to legacy data center environments.

Unfortunately, the article contains a number of common misconceptions. As practitioners supporting over 800 cloud environments, we see deployments at every stage of life — from as early as the architecture (planning) phase all the way through to long-duration deployments that have already been subjected to multiple rounds of carefully targeted optimization. In our view, a generalized debate over whether on-prem environments are cheaper to operate than cloud is incredibly simplistic.

Well-architected and well-operated cloud deployments will be highly successful compared to datacenter deployments in most cases. However, “highly successful” may or may not mean less expensive. A singular comparison between the cost of cloud versus the cost of a datacenter shouldn’t be made as an isolated analysis. Instead, it’s important to analyze the differential ROI of one set of costs versus the alternative. While this is true for any expenditure, it’s doubly true for public cloud, since migration can have profound impacts on revenue. Indeed, the major benefits of the cloud are often related to revenue, not cost.

Two common examples of the cloud’s ability to enhance revenue:

Acceleration of time-to-market cycles

The possibility of rapid expansions in infrastructure (within or even across geographies) to capture revenue blooms

The revenue enhancements associated with both can exceed any theoretical cost premiums for cloud by significant amounts, resulting in very attractive returns on investment when these technologies are applied well.

Short-term thinking brings short-term results

An oversimplified counterexample to Wang and Casado’s assertions will make our logic clear. Suppose a private equity firm approaches a manufacturing concern and advises them that they can cut their cost of revenue metric in half by shuttering half of their factory. What happens to production volumes if they follow this advice? What happens to revenue? If the plant was running at or near capacity, their production capacity — and therefore their revenue — would also be cut in half. Now imagine the half of the factory they closed actually had the most productive assembly lines. Their costs have dropped by half, but their revenue will drop by more. This approach may result in some favorable near-term financial results, but investors with longer-term goals are going to take it on the chin down the road when revenue collapses. If an enterprise bails on the cloud to save costs, how might their time-to-market or revenue elasticity be impacted? What opportunities would be foregone? These dynamics must be considered, and that means analyzing ROI, not isolated metrics like cost of sales or cost of goods sold.

The Dropbox repatriation: statistical cherry-picking

What’s more, by extrapolating the results of successful repatriations to the wider ecosphere of cloud consumers, the authors take entirely too many liberties with the notion that one cloud deployment can be easily compared with another from a cost perspective.

The true “cost” of a public cloud is a function of:

The appropriateness of cloud for specific workloads

The architecture

Efficient operation

By definition, the cloud deployments that were successfully repatriated failed along some or all of these dimensions, as directly evidenced by their successful repatriations. But even in cases where the repatriations were deemed successful, it is hardly certain that repatriation was the best option. For example, if a cloud deployment was poorly architected and/or based almost entirely on lift-and-shift workloads, could those workloads have been refactored to cloud-native instead of returned to a datacenter? We have seen savings of 90% and more in such cases. To extrapolate the “savings realized” in “successful” repatriations cases to the wider universe of cloud consumers and thereby conclude that most or all cloud deployments are equivalent failures represents a wholesale backfire of logic. The fact that these deployments were poorly architected or were better-suited to run on-prem hardly means that all cloud workloads are. If the majority of cloud deployments resulted in outcomes this unfavorable, the stampede to the cloud would not have begun and would not be continuing today.

Don’t worry, you’re not wasting more than half of your infrastructure spend

For modern enterprises, the question is not “cloud versus datacenter” but “which workloads for cloud, which workloads for datacenter?” The process steps for analyzing this decision involve asking the following questions:

Which workloads benefit from the elasticity, geo-flexibility, or technological innovation cloud offers? Which workloads can really “take off” if migrated or currently rely on innovative new services only offered in the cloud? These are the best candidates to be run on a public cloud.

Are current or planned workloads architectured to use cloud-native technologies where possible, or are they lifted-and-shifted clones of datacenter infrastructure? If they can be cloned 1:1 in a datacenter, then companies should always consider re-architecting the workload to take advantage of cloud-native technologies. For example, you can move your Hadoop to cloud as is, but we’ve seen identical queries run in BigQuery 73x faster. You could keep running on VMs, but you could save 60% by refactoring into containers. You could stay with your teraflops on CPU, but you can get an exaflop (yes, that’s 1,000,000x faster) on TPUv4.

Is the ROI of infrastructure spend in the cloud being measured and compared to a model of the same infrastructure costs on-prem?  And vice versa?  Regular validation should be carried out to verify that the correct mix of on-prem and public cloud workloads is being maintained. Critically, the ROI analysis must factor revenue opportunity costs of one alternative over the other. For example, if a workload is being considered for repatriation, the model must factor the revenue degradation that would be imposed by eliminating the cloud’s elasticity and thereby slowing time to market, causing stock-outs instead of capitalizing on revenue blooms, etc.

Are best-in-class practices for operating public cloud infrastructure being followed? Has a well-trained and equipped FinOps team been established?

If you’re running large workloads in the public cloud, it’s not time to panic. It’s highly unlikely you are wasting half or two-thirds of your infrastructure costs by running in the cloud without any incremental benefits to show for it. By following the guidelines above, you can ensure that both your cloud and on-prem deployments are successful, without bailing out of one or the other as a result of tunnel vision on cost alone.

As the director of the FinOps group at SADA, Rich Hoyer develops and delivers services designed to help clients monitor, measure and improve the value of their Google Cloud services.

VentureBeat

VentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact. Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

up-to-date information on the subjects of interest to you

our newsletters

gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More

networking features, and more

Become a member

Find your dream job on VentureBeat Careers