I'm curious - do you live in Silicon Valley? Because I see self-driving cars so often here that it's a non-event, and half the time, there's nobody touching the steering wheel. My former boss has ridden in one, and there were signups for an employee beta-test program before I left Google a year ago. I could easily believe that they will be in the hands of consumers by 2020, and I could believe they'll cost $40K once the sensors they need are in volume production. The technology's a lot more advanced than the handful of press articles make you believe.
> Because I see self-driving cars so often here that it's a non-event
No, you don't. You see cars which have a hands-free driving mode. Every single one of them still absolutely requires a driver.
I've been closely involved in automated transport for years, and the current hype cycle is getting exhausting. There are several orders of magnitude of difficulty between a car which allows drivers to take their hands off the wheel some of the time (or even most of the time), and a car which does not require a driver. We are many years, and possibly decades, away from the latter.
We are certainly still in the "Every single one of them still absolutely requires a driver." phase. Given that you've been in the automated transport field for years, and I haven't, I won't make any comments about the many years/decades timeline, except to say that, in general, the pace of technological growth in many areas has surprised me.
This could be solved by hooking the car sensor platform to a remote controlled driver for the times when hands off operation is impossible. Uber could hire their top 5% of drivers to work at at Uber HQ to be drivers on an as-needed basis. and save on 95% of their labor costs.
Google's goals, at least, are for 100% driver-free cars. Not something where you can take your hands off the wheel for 10 minutes, but one where there is no steering wheel at all. I'll let the company speak for how close they are to that, when they choose to.
The reason I believe this problem won't be solved any time soon is that I don't live there, though I have in the past. It's very easy for someone who commutes along hwy 280, for example, to imagine an autonomous vehicle making the same drive instead. It's a bit more of a stretch, but still within reason, to assert that you could do the same on the Central Expressway or Zanker or places like that.
Not all environments are as friendly. Downtown San Francisco will be extremely hard to get right -- note that I say get right, not dump a robot car there and hope for the best. There are going to be a lot of crushed pedestrians and damaged parked cars, and I'm not sure the people who live there will value having autonomous vehicles enough to overlook that series of incidents. Then there are suburban and rural environments with their own challenges: poor mapping, roads that aren't obviously roads, one-lane roads, poor visibility, lack of signage, and so on. Not every location has weather like the Valley, either; suppose an autonomous car gets stuck in snow. A human driver can usually get out, possibly with the aid of a shovel, carpet, sand, etc. The robot can't do those things, and the passengers may not be able to (for example, it's unlikely that a robotic taxicab would have any provision for passengers to control it).
The world is a much bigger place than the Valley, and comes with much bigger challenges. That's why I don't see any real likelihood that fully autonomous vehicles will be developed any time soon. It's easy to get lost in narrow thinking when your entire world consists of a single place and a bunch of people all living in that same place practicing groupthink. This problem is much more difficult than you imagine.
The valley's not the only place I've ever lived. I grew up in Boston, which certainly has its share of inclement weather, one-lane roads, roads without lane markers, one-way streets, 3 left turns that don't make a right, lack of signage, etc. All of these are challenges, but they're not insurmountable challenges.
I'm pretty sure I've seen self-driving cars on Highway 1 in California, for example, which has construction, cattle gratings, one-lane segments, a lot of twists and turns, pull-offs, and places where it gets very narrow and you have to pull off and let opposing traffic pass. They had safety drivers, but I think this means that Google's at least considering these driving conditions in their software.
All I can say is "we'll see". I personally don't believe these challenges will be addressed in 5 years. Perhaps you do (not putting words in your mouth, since you never claimed this), though we agree that no challenge is insurmountable. 50 years is a good marker here. The PCH is tough but it represents but one slice of the challenge; how about hwy 130 to Mt Hamilton? In January? The idea of a car without a steering wheel is far off to me, even in California.