Very true. I don't even use that feature - I just have my TV cycle through all my Favourite'd photos. It then selects a new subset of those photos to cycle through every day or so. It's perfect, and I get to control which photos don't pop up.
I wish we had more control over the refresh period. It really seems to stick on some that keep coming up for years. I would like to allocate more ram or disk to the screen saver cache.