Hosting Providers for AI

TPU providers, GPU providers, Cloud GPU, etc

Although my initial assumption was that there were not many true GPU or TPU hosting companies out there, further investigation proved this to be incorrect. In addition to the plethora of “big” cloud providers offering their expensive services, the market is also populated by smaller, leaner providers.

• OVHcloud offers GPU cloud solutions.
• Exoscale provides GPU-based cloud computing services.
• Cudo Compute serves as both a GPU cloud provider and a marketplace, appearing quite exceptional.
• CoreWeave is another option for GPU hosting.
• RunPod allows you to rent cloud-based GPUs.
• Lambda Labs GPU Cloud delivers GPU cloud services.
• Fuga Cloud is a Dutch GPU cloud provider.

A comprehensive overview of the entire GPU hosting offerings, along with pricing, can be found at Cloud GPUs.

Vast.ai is a marketplace for renting GPUs and is the proper GPU cloud alternative with more affordable pricing. According to community feedback, it works quite well – see the discussion on Consumer GPU cloud rental at vast.ai.

A practical example of using vast.ai is explained here: course-v3/start_vastai.md.

Vast.ai is renowned but not the sole GPU marketplace provider, as there are also completely open-source community projects with the same concept, such as Golem or Sonm.

Linode, now Akamai offers Cloud GPU compute instances, but they are prohibitively expensive, starting at $1000/month and up.

Paperspace boasts the apt slogan “Cloud computing, evolved” and is the only provider with dedicated GPU hosting at more reasonable, albeit still high, prices. A dedicated GPU costs $300/month, while shared GPUs are available for as low as $22/month. They offer not only Nvidia but also Graphcore TPUs, which they call IPUs, and the prices are surprisingly reasonable – see Graphcore x Paperspace.

Articles on the topic:

• Best Values in Deep Learning Cloud Providers
• Deep Learning CPUs, Cloud Providers, Platforms
• Which are the most cost-effective platforms to train AI models?

Build Your Own AI

At present, it is wiser to construct one’s own TPU and GPU computer, should the need arise. The professional Nvidia Tesla A100 Ampere represents the gold standard for AI, with a new unit costing around $8500—an amount that is not entirely unattainable. Ampere is the code name for Nvidia’s A100 generation of chips, while Nvidia Hopper denotes the forthcoming generation, encompassing the H100, which has recently commenced distribution. Desktop GPUs, such as the RTX 3090, cost around $1500 and exhibit a performance level commensurate with the A100, while the Nvidia Quadro RTX A6000 is available for approximately $4000.

The most optimal benchmark results can be found on the GPU Performance Benchmarks page, as well as the GPU Benchmarks for Deep Learning site, where it becomes evident how much faster the RTX 3090 is, even when compared to the previous generation Nvidia Tesla V100 Volta GPU.

Building Your Own Deep Learning Computer is 10x Cheaper Than AWS held true in 2019 and is even more accurate today. There is also an astonishing discussion on Hacker News from an earlier date, 2018.

It appears that in the past, you could obtain GPU instances at much more affordable rates from Hetzner and OVH. Regrettably, Hetzner ceased offering GPU servers, possibly due to an inability to acquire GPUs.

However, the crucial information from the first text above is as follows: “… AWS is costly because Amazon is compelled to utilize a significantly more expensive GPU. They are not employing the 1080 Ti / 2080 Ti / Titan cards, as Nvidia contractually forbids the use of GeForce and Titan cards in data centers. Consequently, Amazon and other providers must rely on the $8500 data center version of the GPUs and charge a substantial sum for renting it.”

Izuzetno jednostavno i profesionalno Salad - GPU Cloud | 10k+ GPUs for Generative AI