I am prototyping a project using airbyte to replic...
# replication-troubleshooting
g
I am prototyping a project using airbyte to replicate data from MSSQL --> Bigquery. Running airbyte open source on a GCP VM, with an external Google Cloud SQL Postgres database. On every sync, the first attempt fails but the second attempt succeeds. From what I can see in the logs, the issue on the first attempt creates the following error message:
Copy code
2022-10-27 14:40:35 INFO i.a.w.g.DefaultNormalizationWorker(run):95 - Normalization summary: io.airbyte.config.NormalizationSummary@6d415f1a[startTime=1666881618122,endTime=1666881635122,failures=[io.airbyte.config.FailureReason@59d118f6[failureOrigin=normalization,failureType=system_error,internalMessage=Model 'model.airbyte_utils.custom_reports_stream' (models/generated/airbyte_tables/airbyte/custom_reports_stream.sql) depends on a source named 'airbyte._airbyte_raw_custom_reports_stream' which was not found,externalMessage=Normalization failed during the dbt run. This may indicate a problem with the data itself.,metadata=io.airbyte.config.Metadata@3c78dc3b[additionalProperties={attemptNumber=0, jobId=30, from_trace_message=true}],stacktrace=AirbyteDbtError: 
Encountered an error:
Compilation Error in model custom_reports_stream (models/generated/airbyte_tables/airbyte/custom_reports_stream.sql)
  Model 'model.airbyte_utils.custom_reports_stream' (models/generated/airbyte_tables/airbyte/custom_reports_stream.sql) depends on a source named 'airbyte._airbyte_raw_custom_reports_stream' which was not found,retryable=<null>,timestamp=1666881635122], io.airbyte.config.FailureReason@744c05a7[failureOrigin=normalization,failureType=system_error,internalMessage=Model 'model.airbyte_utils.custom_reports_stream' (models/generated/airbyte_tables/airbyte/custom_reports_stream.sql) depends on a source named 'airbyte._airbyte_raw_custom_reports_stream' which was not found,externalMessage=Normalization failed during the dbt run. This may indicate a problem with the data itself.,metadata=io.airbyte.config.Metadata@13123dbb[additionalProperties={attemptNumber=0, jobId=30, from_trace_message=true}],stacktrace=AirbyteDbtError: 
Encountered an error:
Compilation Error in model custom_reports_stream (models/generated/airbyte_tables/airbyte/custom_reports_stream.sql)
  Model 'model.airbyte_utils.custom_reports_stream' (models/generated/airbyte_tables/airbyte/custom_reports_stream.sql) depends on a source named 'airbyte._airbyte_raw_custom_reports_stream' which was not found,retryable=<null>,timestamp=1666881635122]]]
1. Can someone help me understand this error message? 2. Also, since it works every time on the second attempt… is the second attempt doing something different that allows it to succeed? Maybe it’s simply skipping the normalization process, or something like that?
s
Hey there, this looks like Airbyte is trying to normalize the synced data in your destination, but it isn't able to find the raw tables. It's likely that something went wrong with the sync itself, do you have the full sync logs including from before this error? Also if you have the full sync logs for the successful attempt, that would be helpful too! Can't say exactly why until we see more logs, but the reason why it works on the 2nd try is likely because the sync was able to complete. This means we can probably rule out access issues or a normalization bug, as Airbyte was able to eventually sync and normalize the data (assuming your destination data is in an expected shape)