Calculus is not just about gradients. Fundamentally, calculus is about calculation and symbolic logic, a much older concept that predates its usage in gradient-based machine learning. In this sense, calculus holds deep connections to logical reasoning, proof theory and the foundations of computer science. [1]
This. The importance of learning calculus and model building (physics) is all about learning to reason, making predictions and quantify domain validity. It's not about calculating derivatives or proving continuity. I think people take these requirements too literally.
[1]: https://compcalc.github.io/