Hello, anyone here is using JdbcSink with Aurora P...
# randoma
Anshum Verma
03/15/2023, 9:10 PMHello, anyone here is using JdbcSink with Aurora Postgres. The production load is expected to scale upto 90000 records per minute. the performance tests reveal that with 120 parallism and average 750 records per minute from each slot is not an issue. But I would like to know if any gotchas are missed before this is deployed
s
sap1ens
03/15/2023, 9:20 PMYep, I have a job with parallelism of 4 doing ~10x of your rate when writing to Aurora Postgres
sap1ens
03/15/2023, 9:20 PMUse the following JDBC URL parameters if you can:
Copy codeassumeMinServerVersion=10.0&reWriteBatchedInserts=true
a
Anshum Verma
03/16/2023, 1:41 AMWow with a parallelism of just 4. Seems like you have large sized nodes. Thanks for suggesting the rewrite configurations. I completely missed out using that