The numbers in the blog are 350 rps for 48 vCPUs (hyper threads), which is 24 cores.
That's just a hair over 7 requests per hyper thread per second, or 135ms per request. That second number can be seen in the last graph, where the last graph shows an average request latency of about the same value, I'm guessing 120 milliseconds...
Without knowing what the code is doing, it's unfair to judge, but you've got to wonder what Netflix does to burn through $1B-per-annum in cloud infrastructure expenditure. Consider that at that rate they must have a truly epic deep discount, so that's probably equivalent to $3 to $5 billion at on-demand or retail pricing. Bonkers!
Wow that feels slow. 135ms smells like their blocking heavily for some kind of database I/O and not processing anything else. But surely that can’t be the case. They invested a lot in funding a cache line bouncing issue in the JVM so this service must be fairly valuable that you’d expect lower having fruit like that to have already been solved. But the only compute intensive parts of their business I can think of would be transcoding (not 135ms and it would be native code in the background), maybe pre-rendering initial UI, and processing analytics (also background). My guess is maybe on the pre-rendering (not sure if they do that) but 135ms feels steep when you consider what games accomplish in 10 (sure - not quite the same but 135ms would be an eon to accomplish something in). I am kinda curious what this service is responsible for now. A 7 RPS service that’s this valuable seems interesting.
That's just a hair over 7 requests per hyper thread per second, or 135ms per request. That second number can be seen in the last graph, where the last graph shows an average request latency of about the same value, I'm guessing 120 milliseconds...
Without knowing what the code is doing, it's unfair to judge, but you've got to wonder what Netflix does to burn through $1B-per-annum in cloud infrastructure expenditure. Consider that at that rate they must have a truly epic deep discount, so that's probably equivalent to $3 to $5 billion at on-demand or retail pricing. Bonkers!