You will for sure, I actually like it but not everyone does. You can remineralize and change the taste without too much trouble. I would avoid any other claims made for that process but it does usually improve the taste. Straight RO water is not great for coffee or tea as well.
Some grocery stores and walmart's have RO water machines outside their stores also. However, if you choose to use those it would be good to invest in a tester as they do not always change the filters on schedule.