Yeah, if you have a fixed llm topology, you can just effectively burns 2 top layers of the chip as Rom (model weights) - which has a per area density even better than dram - so it’s just attention and kv streaming that is hbm to sram transfer.

Most big model weights will not fit a single reticle sized chip - so you’d have prob 30 different chips to split the model .

And you’d need super fast chip to chip comms for the all-reduce and similar.

So scaling to 1T models is hard - and a long lead time - but can be very power efficient.