Hi team, i need some help understanding what middleware is used by airbyte to communicate between two connectors? Could anyone please help me out with this? Is there any documentation for this? I am using the open source verisosn of Airbyte

Hi. What do you mean by communicate between two connectors exactly?

Hey @tanuj soni, here's a high level overview of Airbyte's architecture: https://docs.airbyte.com/understanding-airbyte/high-level-view We've got lots more in the docs, so I encourage you to explore them. I second Eli - could you clarify your question? That would be really helpful!

I wanted to know how the source connector sends data to the destination connector, what is the exact middleware being used? I saw the architecture document and found out it uses temporal which manages the queues.

The temporal does:

- checkout master
- build the docker image
- run docker compose up in the background
- Sleep for 75 secondes
- shutdown docker compose
- checkout the commit being tested
- build the docker image
- run docker compose up

The workers:

Are Jobs that are executed. Jobs are executed using a tool called Temporal.

are you asking about how does airbyte takes data from the source and pushes to the destination connectors?