If the content is not truly infinite (#comments known?), just very large, you could solve the scrollbar issue by giving the page the right vertical size (or an estimation), but only populating the HTML if the user scrolls somewhere.

so the page looks broken or like its still loading? Yah great usability

You'd use AJAX to load the elements, so it doesn't show as loading. And you make sure the user never sees the empty space, given normal scrolling speed, by loading the area around the visible screen. Certainly not worse than extending the page when the user reaches the bottom, as some sites do.