We would definitely seriously consider switching to GCS more if your cloud functions were as powerful as AWS Lambda (trigger from an S3 event) and supported Python 3.6 with serious control over the environment.
Is there something about the GCS trigger that doesn't work for you? I hear you on Python 3, but I'm also curious about "serious control over the environment". Can you be more specific?
Here are our main issues with Lambda, from highest-to-lowest priority:
- It supports Python 2.7 only. We need Python 3.4+ support.
- We can't increase CPU allocation without increasing RAM allocation, making them far more expensive than we need.
- Using psycopg2 on it is a PITA due to their handling of system dependencies.
- The system is entirely proprietary, making it impossible to run it locally for testing.
- Cloudwatch sucks for finding errors in the functions and is atrociously expensive.
- API gateway is an extremely crufty system, and used not to let you pass around binary data (this has changed)
- We can't disable/change the retry-on-error policy.
We have a pretty hard tie-in to S3 and Redshift, but when GCF can do better on a majority of these points, we'll begin moving to it. But yes, Python 3 at a minimum would be a requirement.
> The system is entirely proprietary, making it impossible to run it locally for testing.
I assume that you are referring to emulating the triggering of lambdas behind API gateway...? I've found a project that sets up a node environment to do this. Very handy for js/lambda development. A google search suggests similar options may exist for python.
As someone who's literally just starting to look at Lambda, thanks for that quick read.
I had a lot of "chicken and egg"-type questions about using it, and seeing that critical step of bootstrapping the whole thing via the API Gateway was really informative.
