> - Testing frameworks and "best practices" change way faster than language frameworks and I simply can't keep up. What rspec version do I use with what version of Rails? Now I have to use Cucumber? I learned some Cucumber ... oh, now Steak is better. [rage comic goes here]
I think this is only in the Rails community, where all new things is quick to be pronounced "the new right way to do things", not just in testing.
> I deal with custom domains, authentication, and 3rd party API calls in almost every app we have. IMO, this adds 20% or more to the (already high) testing overhead just to get these things configured right in the test suite
We do the same (tests for interaction with EC2, Github, and a few other providers). It is more expensive, but we find it more worthwhile too. Normally, 3rd party APIs are insufficiently specified, especially for error conditions. So when we have a failure in production, we can easily add tests to make sure we handle that edge case in future.
I think this is only in the Rails community, where all new things is quick to be pronounced "the new right way to do things", not just in testing.
> I deal with custom domains, authentication, and 3rd party API calls in almost every app we have. IMO, this adds 20% or more to the (already high) testing overhead just to get these things configured right in the test suite
We do the same (tests for interaction with EC2, Github, and a few other providers). It is more expensive, but we find it more worthwhile too. Normally, 3rd party APIs are insufficiently specified, especially for error conditions. So when we have a failure in production, we can easily add tests to make sure we handle that edge case in future.