If agree in theory, but I'd like to disagree with some assumptions. AI devs don't necessarily have experience with efficient network infrastructure. (And ops people with AI development)
You can't just "buy servers for the same price". Where are you going to put them? How are you going to power them? How is the bandwidth for them provisioned? At some scale these are non-trivial questions - you can't just buy a rack, stick it next to your desk and plug into an extension cord.
The system deployment is something you need to spend time on as well. Bare metal provisioning and deployment of GPU libraries to make things run smoothly takes time.
And finally when the hype dies down in a week or two, what are you going to do with that infrastructure?
Cloud services are not trivial either. But they do have some advantages.
Possibly the author of the parent comment was puzzled why does an AI lab not have a specialist who provisions the bare metal. I would think that if the hype died in a week, an AI lab would have other projects which require similar equipment to run.
I agree with your post in general as intuition tell me (without further details about ops situation) that cloud is a competitive fit in this scenario.
At this "scale" a single 3Gbps guaranteed pipe is ~330€/mo on OVH. You could simply buy ten of them just to be sure and still be paying 1/3 a month of what that would cost you for a day in the cloud. That includes all the maintenance and power for the equipment, you simply have to know how to use the OS.
You can't just "buy servers for the same price". Where are you going to put them? How are you going to power them? How is the bandwidth for them provisioned? At some scale these are non-trivial questions - you can't just buy a rack, stick it next to your desk and plug into an extension cord.
The system deployment is something you need to spend time on as well. Bare metal provisioning and deployment of GPU libraries to make things run smoothly takes time.
And finally when the hype dies down in a week or two, what are you going to do with that infrastructure?
Cloud services are not trivial either. But they do have some advantages.