Agreed. I started tinkering in this space over the last year (done loads of software, started trying to make my own synth). The lack of an oscilloscope basically makes it impossible to get anything done - you're kinda just working on hunches.
People kept recommending the RIGOL DS1054Z, but at £300 or so it's too punchy to invest in unless you know you're really going to get a chance to use it. Will definitely try one of these little units.
For audio and most 8 bit work you can buy a two channel 10Mhz picoscope for about £120. Really solid piece of kit, and it's designed in the UK (if that has any weight). Has tons of stuff like signal decoding built into the UI which you have to pay for (or hack) on the Rigol. Don't believe the crap about USB scopes being bad. It's really handy for not taking up valuable bench space, and you can take it in the field easily. It also has a basic signal gen, but I've not used it much.
I also have a 1054 which is great - sometimes you just need 4 channels. I would be happy with either.
The trade off is something you can fit in your pocket versus two extra channels and more bandwidth. The 4 channel 50Mhz version is actually quite close to the Rigol price.
I have the 2204A and can second the recommendation. Really good bang for the buck (120EUR here in Germany, including two probes), very compact and robust. Of course, with 10MHz you shouldn't expect too much, but for looking at low-speed communication it is perfectly OK. What I really enjoy are the many different trigger options (and they actually work).
When buying an USB scope one must remember that it is nothing without the software, and the one from Picotech is well done. They also support Linux although that one is still in Beta and still has less features, but they say they are working on feature parity. It's not (F)OSS, though.
Amateur tip though, lots of good deals on used scopes out there. Companies seem to cycle through them, probably a calibration thing. I've seen 200MHz scopes go for < $150 USD on Ebay in the past, (inconveniently enough at times when I didn't have $150 to blow for an older Tektronix).
People kept recommending the RIGOL DS1054Z, but at £300 or so it's too punchy to invest in unless you know you're really going to get a chance to use it. Will definitely try one of these little units.