The demand for cloud alternatives has never been greater.
Case in point: CoreWeave, a GPU infrastructure provider that started as a crypto mining operation, raised $1.1 billion in new funding this week from investors including Coatue, Fidelity, and Altimeter Capital. This round values Post-Money at $19 billion and brings total debt and equity raised to $5 billion. This is a surprising number for a company that was founded less than 10 years ago.
CoreWeave is not alone.
Lambda Labs, which also offers a range of cloud-hosted GPU instances, raised up to $500 million in “special purpose financing vehicles” in early April, months after closing a $320 million Series C round. Secured. Voltage Park, a nonprofit backed by cryptocurrency billionaire Jed McCaleb, announced last October that it would invest $500 million in GPU-powered data centers. And Together AI, a cloud GPU host that also does generative AI research, raised $106 million in a Salesforce-led round in March.
So why is there so much enthusiasm and money pouring into alternative cloud spaces?
As you might have guessed, the answer is generative AI.
As the generative AI boom continues, so will the demand for hardware to run and train generative AI models at scale. Architecturally, training, fine-tuning, and running a model requires a logical It will be a choice.
However, installing a GPU is expensive. That’s why most developers and organizations are turning to the cloud instead.
Cloud computing incumbents Amazon Web Services (AWS), Google Cloud, and Microsoft Azure all offer GPUs and specialized hardware instances optimized for generative AI workloads. But cloud alternatives may end up being cheaper and more available, at least for some models and projects.
CoreWeave rents an Nvidia A100 40GB (one of our popular choices for model training and inference) for $2.39 per hour, which translates to $1,200 per month. On Azure, the same GPU costs $3.40 per hour, or $2,482 per month. Google Cloud costs $3.67 per hour or $2,682 per month.
Given that generative AI workloads typically run on clusters of GPUs, the cost difference quickly increases.
“Companies like CoreWeave are participating in what we call a market of specialized ‘GPU as a service’ cloud providers,” Sid Nag, vice president of cloud services and cloud technologies at Gartner, told TechCrunch. Told. “Given the high demand for GPUs, they are offering an alternative to hyperscalers, where they take Nvidia GPUs and provide another route to market and access to those GPUs. doing.”
Nag notes that even some large tech companies are facing computing power challenges and are turning to alternative cloud providers.
Last June, CNBC report Microsoft says it has signed a multibillion-dollar deal with CoreWeave to ensure OpenAI, the developer of ChatGPT and a close Microsoft partner, has enough computing power to train generative AI models. His Nvidia, which supplies the majority of CoreWeave’s chips, sees this as a desirable trend, perhaps for leverage reasons.Said to have offered several alternative cloud providers priority access to the GPU.
Lee Sustar, principal analyst at Forrester, says cloud vendors like CoreWeave are successful in part because they don’t have the infrastructure “baggage” that incumbent providers have to deal with. I think there is.
“Given the dominance of hyperscalers across the public cloud market, challengers like CoreWeave have no choice but to invest in hyperscalers, which require huge investments in infrastructure and various services with little or no return. There is an opportunity to succeed by focusing on premium AI services without the burden of scaler-level burdens,” he said.
But is this growth sustainable?
Mr. Suster has his doubts. He believes the growth of alternative cloud providers will depend on their ability to continue to bring GPUs online in large quantities and at competitively low prices.
Price competition may become more difficult in the future as incumbents such as Google, Microsoft, and AWS increase their investments in custom hardware to run and train their models. Google offers his TPU. Microsoft recently announced two of his custom chips: Azure Maia and Azure Cobalt. AWS has Trainium, Inferentia, and Graviton.
“Hypercaler will leverage custom silicon to reduce dependence on Nvidia, while Nvidia will look to CoreWeave and other GPU-centric AI clouds,” Sustar said.
Additionally, while many generative AI workloads run best on GPUs, not all workloads require GPUs, especially for time-independent workloads. A CPU can perform the required calculations, but is typically slower than a GPU or custom hardware.
From a more existential perspective, there is a threat that the generative AI bubble will burst, leaving providers with too many GPUs and few customers to demand them. But in the short term, the future looks rosy, said Suster and Nag, who both expect the start-up cloud to steadily roll in.
“GPU-oriented cloud startups [incumbents] “There is a lot of competition, especially among customers who are already multicloud and can handle the complexity of management, security, risk and compliance across multiple clouds,” Suster said. “With trusted leadership, solid financial support, and zero-latency GPUs, these cloud customers can confidently try new AI clouds. ”