Yes, it is possible to use a dedicated database for Airbyte's metadata. You can manage the deployment of the Airbyte database in two ways: 1. Using the default Postgres database that Airbyte spins up as part of the Docker service described in the

docker-compose.yml

file:

airbyte/db

. 2. Through a dedicated custom Postgres instance (the

airbyte/db

is in this case unused, and can therefore be removed or de-activated from the

docker-compose.yml

file). It's not a good practice to deploy mission-critical databases on Docker or Kubernetes. Using a dedicated instance will provide more reliability to your Airbyte deployment. Moreover, using a Cloud-managed Postgres instance (such as AWS RDS or GCP Cloud SQL), you will benefit from automatic backup and fine-grained sizing. You can start with a pretty small instance, but according to your Airbyte usage, the job database might grow and require more storage if you are not truncating the job history [source](https://docs.airbyte.com/operator-guides/configuring-airbyte-db). Regarding memory usage, you can try increasing the memory allocated to your Airbyte server, as it may improve the batch size and help with the connector's performance [source](https://airbytehq.slack.com/archives/C021JANJ6TY/p1663179159558039). Additionally, you can review the [Scaling Airbyte](https://docs.airbyte.com/operator-guides/scaling-airbyte) guide for more information on managing memory usage. If you're still experiencing issues, consider creating multiple connections for different tables that all sync to the same destination, as this would parallelize the syncs at a connector level [source](https://discuss.airbyte.io/t/1977).