Neural nets as an abstract concept don’t necessarily need to be built with atoms, just the substance of the external universe which may be far simpler.
I also didn't imply this neural network's universe needs atoms, but it needs to run the computation somehow, the computation itself is the complexity, no matter with what physics or other manifestation it shows up
And if it is simulating our universe's atoms, then that part of it basically is our atoms. But is a neural net doing that really simpler than the atoms just running with a more direct mathematical model?
You don’t need atoms, just a universe with the minimal blocks to build a turning machine. Like building a computer in Minecraft they could conceivably run these neural nets without any sense of ‘atoms’. The point is the outside universe could have extremely simple design.