SoundCloud uses a microservices architecture. A microservices architecture inevitably comes with additional complexity, among which the evolution of APIs in a backwards-compatible way.
To change (or remove) an endpoint, the consumers of that endpoint need to be known. This doesn’t come out of the box.
To solve this, I introduced an Sc-System HTTP header which contains the name of the service making the call. It identifies the caller. Analogous to a phone call: “Hey service Bar, this is service Foo calling on behalf of whomever/whatever.” In this case, the HTTP call would include Sc-System: foo.
Each service at SoundCloud had its own repository, and the name of the system was chosen to be identical to the repository name.
Some services made the Sc-System header mandatory.
We then tracked incoming HTTP requests with a dimension for the caller (Sc-System), and a dimension for the HTTP path pattern (e.g. /tracks/:id), and so we could see which clients were using a specific endpoint, and work with the teams owning those clients to ensure backwards-incompatible changes are handled gracefully.
Related:
-
Distributed tracing can likely also be used to solve this problem, but it’s a much more heavyweight solution.
-
Client certificates can likely also be used to solve this problem.