When I was going through flight training, I thought about this a lot. I think the reason is that the places airplanes go is not random. If airplanes just went to random places in the sky at random altitudes, then indeed accidents would be rare, but everyone heads towards airports, which have particular traffic pattern altitudes for instance. I don't know the specifics of this situation, but perhaps the military pilot wanted to practice IFR near the ground and there also happened to be a VFR corridor passing through there. These non-random intents of pilots mean that airplanes get a lot closer together than you would otherwise think.
There's a known problem in the sailing world where people will set their autopilots to GPS coordinates from waypoints published in sailing magazines, and then collide with another vessel in the middle of deep ocean with no landmarks around.
