The solar radiation at the poles is much smaller than at the equator, so the contribution of polar ice to the total radiation-weighted albedo of the planet is much smaller than the contribution of equatorial deserts.
When the radiation happens, the sun ain’t shining anyway. What matters is the surface temperature, and the difference doesn’t look too bad, around 15%: 300°K in tropics, 250°K in arctic.
High albedo is important because of its effect on the net reflectivity of the planet, not its effect of the radiation of the surface.
Emissivity is the material property involved in radiation loss, and ice is highly emissive, but so is almost anything non-conductive. Also radiation loss depends on T^4, so that 15-20% difference ends up being closer to one being double the other.