Everyone already pays the same operational burden with their house keys, which are far more difficult to manage for the average person (as they leave the house constantly). It's worked fine for hundreds of years.
If you lose your house keys, you get a lock smith to break into your house for you - your house doesn't become unusable forever more.
If you ever need to have this done, you'll realise how much house keys and door locks for many cases really only stop the opportunistic "pull the handle and see if it opens" attack. If your door has above average security they'll need to drill the lock, but the time I had to call one they could just push a tool through the letter box and break/move the bolt by applying leverage from the "indoor" side.
