spotify: handle rate limit
Spotify has its own rate limit policy: https://developer.spotify.com/documentation/web-api/guides/rate-limits/
We found ourselves quickly hitting this limit when lots of users when searching and adding songs together. The main culprit seems to be retrieving the covers of the cdg songs as each search may trigger a lot of requests (one per cdg song found in the db). We already have a soft cache but that does not seem to be enough.
Spotify does not document the actual metric of the rate limit but according to https://medium.com/mendix/limiting-your-amount-of-calls-in-mendix-most-of-the-time-rest-835dde55b10e it's about 180 requests per minute.
Some ideas:
- Store the cover URL in the DB, but it's unclear how persistent cover URLS are. Maybe we should even download the cover and cache it locally?
- have our spotify client count the number of requests it sent over this 30 seconds window and stop fetching covers after a specific threshold. We could also prioritize search requests over covers.
- handle the
Retry-after
field when getting a429
reply. - request a rate quota extension (required to be an official approved app)