Hello
Recently I downloaded Airbyte on my local m...
# ask-community-for-troubleshootings
Sami RIAHI
04/05/2022, 3:48 PMHello
Recently I downloaded Airbyte on my local machine and I made some modifications to the code (for example the "setFetchSize" and I increased from 1K to 1M ), I increase also JOB_MAIN_CONTAINER_MEMORY_REQUEST=20g, JOB_MAIN_CONTAINER_MEMORY_LIMIT=26g ,After that I deployed Airbyte on Docker and I launched the synchronization but I do not see changes in the logs.
can you please help me ?
✅ 1
m
Marcos Marx (Airbyte)
04/05/2022, 9:53 PMYou need to re-build the connector you want to use it, and cahnge the version to dev in the UI
s
Sami RIAHI
04/06/2022, 9:09 AMAfter increasing the size of the "setFetchSize", I launched the command "docker-compose up".
How i can do the re-build and how i change the version to dev in the UI ?
please can you detail the steps.
Sami RIAHI
04/09/2022, 12:10 PMHi @Marcos Marx (Airbyte)
please can you detail the steps.
Sami RIAHI
05/09/2022, 5:34 PMdoes there is any limit for the FetchSize, i can increase more than 5K ?
m
Marcos Marx (Airbyte)
05/09/2022, 5:57 PM@Sami RIAHI last week we submit an update where airbyte will batch size dynamically
Marcos Marx (Airbyte)
05/09/2022, 5:57 PMdo you want to try it?
s
Sami RIAHI
05/10/2022, 2:22 PMHi @Marcos Marx (Airbyte)
If you talk about this https://github.com/airbytehq/airbyte/pull/12400, yes i see it and i will use it.
Now if i increase the capacity of my database sources (increase the buffer size and fetch size ) and if i give 20GO of RAM to Airbyte (JOB_MAIN_CONTAINER_MEMORY_REQUEST and JOB_MAIN_CONTAINER_MEMORY_LIMIT ) does this will help me to have better performance ?
however the preformance the logs will print every 1000 rows ?
4 Views