[AWS Aurora MySQL /rdsdbdata/tmp is full]
# ask-community-for-troubleshootingQuestion
How can I resolve the 'The table is full' error when trying to replicate data from MySQL to BigQuery?
Answer
One possibility is to clear the temporary tables and try to re-run the sync. Alternatively, you may need to increase the allocated memory for Aurora. Upgrading the connector to the latest version may also resolve the issue. For improving performance in loading big tables in MySQL, create a separate thread specifically for that issue on the same channel.
m
Maykon Lopes
10/05/2022, 12:12 AM[AWS Aurora MySQL /rdsdbdata/tmp is full]
✍️ 1
Maykon Lopes
10/05/2022, 12:13 AMHey folks, I'm facing an issue to replicate data from a mysql to bigquery, the fail is really quickly in log output, showing this:
Copy code
Caused by: java.lang.RuntimeException: java.sql.SQLException: The table '/rdsdbdata/tmp/#sql129d_35_1' is full
at io.airbyte.db.jdbc.StreamingJdbcDatabase.unsafeQuery(StreamingJdbcDatabase.java:79)
at io.airbyte.db.jdbc.JdbcDatabase.unsafeQuery(JdbcDatabase.java:195)
at io.airbyte.integrations.source.relationaldb.AbstractRelationalDbSource.lambda$queryTable$0(AbstractRelationalDbSource.java:61)
... 34 more
Caused by: java.sql.SQLException: The table '/rdsdbdata/tmp/#sql129d_35_1' is full
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)Maykon Lopes
10/05/2022, 12:14 AMI tried increase size in aurora rds mysql parameter group, but still having the issue. Any clue?
u
user
10/05/2022, 11:47 AM@[DEPRECATED] Marcos Marx turned this thread into Zendesk ticket 2574 to ensure timely resolution!
s
Saj Dider (Airbyte)
10/05/2022, 2:39 PMHey @Maykon Lopes, how much memory did you allocate for the aurora and what is the size of the sync in terms of records and bytes?
My guess is that you can clear the tmp tables and try to re-run the sync. Otherwise, you may not have increased the memory enough
m
Maykon Lopes
10/19/2022, 5:57 PMHey @Saj Dider (Airbyte) sorry my late answer here, the issue was solved after upgrade and connector to lastest version. Thanks for you help.
Maykon Lopes
10/19/2022, 5:58 PMNow I'm struggling to improve performance in load big tables in mysql, tables with more than 300GB.
s
Saj Dider (Airbyte)
10/19/2022, 7:17 PMHey @Maykon Lopes, happy to hear that upgrading the connector fixed your issue. Create a separate thread for the performance issue with MySQL on this channel.
👍 1
310 Views