Hey everyone! I’m currently using Flink 1.20.0, fl...
# troubleshooting
d
Hey everyone! I’m currently using Flink 1.20.0, flink-connector-mongodb-cdc 3.1.1, and flink-bigquery-connector 1.0.0 (by Google Cloud Dataproc). The current architecture is: MongoDB → Flink CDC → BigQuery. When triggering a snapshot from MongoDB, I’m using the DataStream API format from the flink-bigquery-connector with the sink parallelism set to 1. However, the loading speed seems extremely slow—I’m only able to load around 30 million records in 4 hours. Is there any way to speed this up or optimize the setup?