> Once grown, this brain knows nothing about objects in the physical world.

Unfortunately this experiment has already been done, on an abused girl named Genie.

> one can imagine, the know-nothing brain will learn to

She was never able to learn.

So does this refute your premise, or will you say "that's different"?