Run on a single machine by logically partitioning GPUs. Don't get me wrong, I'm not disputing that this could work or that it could be a "breakthrough". I'm just saying that unless it's independently replicated and confirmed, it's just a paper like a million others.
It's an interesting premise nonetheless. Perhaps another similar approach would be one from mathematical manifolds, where they have charts and atlases, and I believe they build the atlas by having overlapping charts.
