https://docs.all-hands.dev/ logo
Join Slack
Powered by
I'll relate the experience we had while making Atl...
# proj-agent-canvas
n
I'll relate the experience we had while making Atlassian's 
Stride
Platform as an analogy .... Stride was Atlassian's nextgen chat platform to replace HipChat. -- it was an amalgam of a bunch of microservice apis interconnected by a nessagebus .... the idea of latency is interesting .... in the context of CHAT .... urgency is crucial .... think about the palpable experience you have when you see the 
*PERSON*  is typing
indicator whilst chatting with someone of significance .... the difference between 50ms and 100ms is palpable ...
so between the time that your chat message entered our front door, and egressed it back out for delivery .... > 70 microservices touched your message content
that had to happen in < 100ms
everything interconnected into kafka queues (not advocating kafka over some other tech ... just providing grounding)
one of the most useful things we made in devops/devex was a canary service that would: • start up • connect to kafka ◦ emit metrics • send a consistently sized message to a queue ◦ emit metrics • disconnect ◦ emit metrics • connect to kafka ◦ emit metrics • consume message from queue ◦ emit metrics • disconnect • emit metrics
having a tool that provided VERY CONSISTENT metrics, that was NOT RELATED TO DEV RELEASES made it a LOT EASIER for teams to validate if the plumbing was broken ..... and allowed for MUCH SMALLER error margins because you weren't including content-size-or-complexity variablility into norm averages
it was also somewhat typical for dev teams to want to blame kafka, as that was the edge of their realm of responsibility
so providing a tool to assist in diagnosing if the plumbing was / wasn't contributing to the problem helped me save my teams from getting called into incidents where we weren't really needed, but had to be present to avoid being blamed
Open in Slack
PreviousNext
Powered by