Exactly, make sure the only copies of the wallets with your money are on non-networked, non-running hardware, preferably just straight up storage media, and secure it physically. You can make it as secure as you can possibly make anything. Hell, you could even get it insured.

Though interestingly, your bank isn't generally required to repay you for losses in your safe deposit box unless it arises from their negligence.

Probably makes sense to print it out on a sheet of paper and put it in there too.