Hi all,
In our system we have to monior and track the number of calls to our API that each user does overtime. Alrought we are fine with the sum of the last ~30 days for billing purposes (thus I could use a redis system with TTL) i would also like to plot the chart of calls overtime, at least for the last day to have monitor insights.
So far we have a system that sends data to `statsd` and then to `graphite`, it works, but it's not easy to port to the other servers, and it requires quite a server to handle all the series (each user has calls, errors and other data), thus it's roughly `n-of-users * 7`. I was thinking of migrating stats to prometheus, but it does no works well with having so many time series (according to their documentation, which is actually written at the end :( https://prometheus.io/docs/practices/naming/ )
I was thinking of logging (ELK/graylog) each call and then run a query over elastic, but it seems a bit of work and it does not seem to be so simple to plot the charts.\
any idea or system to use?