To just continue a bit on

justifications

- am wondering of airbyte team could share a sentence or two on what’s like building a production service using temporal(cadence)? what are the pros/cons vs other alternatives you’ve considered? Thanks a lot!

Happy to! Our choice of Temporal is from experience having to orchestrate hundred to thousands of jobs at the same time. There are two big hurdles with such orchestration 1) scaling state management 2) proper queue management. Temporal solves this by offering primitives that allow serialising the jobs' current runtime memory into a DB. Since a job's entire state is stored, it's trivial to recover from failures and it's easy to determine if a job was assigned correctly. It's also insanely performant compared to the previous set ups we've seen. Pros: does the job really well and is quite cheap for the scale Con: pretty steep learning curve, the primitives aren't exactly straightforward to grasp