SoundCloud uses a microservices architecture. A microservices architecture inevitably comes with additional complexity, among which the evolution of APIs in a backwards-compatible way.
To change (or remove) an endpoint, the consumers of that endpoint need to be known. This doesn’t come out of the box.
To solve this, I introduced an
Sc-System HTTP header which contains the name of the service making the call. It identifies the caller. Analogous to a phone call: “Hey service Bar, this is service Foo calling on behalf of whomever/whatever.” In this case, the HTTP call would include
Each service at SoundCloud had its own repository, and the name of the system was chosen to be identical to the repository name.
Some services made the
Sc-System header mandatory.
We then tracked incoming HTTP requests with a dimension for the caller (
Sc-System), and a dimension for the HTTP path pattern (e.g.
/tracks/:id), and so we could see which clients were using a specific endpoint, and work with the teams owning those clients to ensure backwards-incompatible changes are handled gracefully.
Distributed tracing can likely also be used to solve this problem, but it’s a much more heavyweight solution.
Client certificates can likely also be used to solve this problem.