I made the same decision on my personal blog. When there are comments both on a blog and here on HN, the conversation feels unnaturally fragmented to me.
Further, when I did have both, comments that self-selected the blog (vs HN) are - in my opinion - much more likely to be spam/promotional, or generally less 'valuable' (e.g. more likely to be simple questions that are already answered in the post).
There's an angle I hadn't considered. The spammy and promotional type of comments would be far more likely to appear directly on the blog since that's where the value (for the commenter) is in leaving them. That's something that moderated comments would control, but you can't moderate your way out of fragmentation.
I would expect more negative comments to be left directly on the blog as well since the commenter won't have to deal with the potential backlash from their community. Which is probably also a good reason to never allow any kind of anonymous of guest comments. (Although that wouldn't stop the truly motivated negative commenters.)
Moderation can also get tricky. Pure spam is simple but you will run into comments where the commenter somehow works in their (product || experience). So long as they provided some commentary it is not pure spam. So the situation becomes a bit touchy.
This is a problem we ran into at PhpPhotoUploadr. In our case we have a very talented team so we hacked together a solution.
Further, when I did have both, comments that self-selected the blog (vs HN) are - in my opinion - much more likely to be spam/promotional, or generally less 'valuable' (e.g. more likely to be simple questions that are already answered in the post).