how much of that do you really use for ML

Not much of your calculus book will be relevant beyond the first couple of chapters, but you'll live and die by the numerical-method sword. The idea is that you need the analytic insight from the former to understand the latter.

I don't know if I really buy that, though.