Egregoros

@lattera the density is higher i think. I've heard that they can't just swap out hardware if they want to upgrade. Requires entire tear down of the racks, power, and cooling and rebuilding it.


edit: obviously this just sounds insanely stupid and risky to not be able to repurpose the infra

@feld @lattera Basically yes as far as I understand it. The equipment comes in full racks, usually custom, and fully wired internally. And since nVidia, AMD and OCP all have different rack sizes, it probably requires an almost full teardown at best.

@phnt @lattera @feld iirc are there a few engineering problems, why retrofitting isn't really possible.
like those racks have A LOT more weight per sqm and A LOT higher Power-Demand then what one would normally plan for with a normal datacenter.
Also probably higher cooling needs with more compute density with all those GPUs(?)

@feld @lattera Companies like Amazon and Google already don’t service systems within a rack. They build a whole rack, deploy it as one unit, then decommission it as one unit when a certain percentage of it has failed.

I suspect what makes an “AI datacenter” is actually buzzword bingo, but it could plausibly be about the dominant type of compute it provides: CPU or GPU.