Hi everyone, I've created source low-code connecto...
# help-connector-development
i
Hi everyone, I've created source low-code connector. On test it works fine, returns all the records - great. But in the connection it returns no errors but in the end it says that there are 0 records. I tried to debug it, connected proxy and it turns out it successfully sents and receives the requests. Why could there be 0 records in the end if the connecto receives them? One thing is that it receives gzipped csv. I handled it in the connector and it gracefully parses it in the test. Another thing - maybe somehow Bigquery destination messes with it? Could you give me a hint, how to debug this further?
k
There’s no information in the provided knowledge sources that directly describes this exact behavior for a low‑code source + BigQuery destination, so I can only infer based on similar “0 records but no errors” cases. From other threads, “0 records read/emitted” with no explicit error usually means: 1. Records are emitted by the source process, but not counted by the worker In the Files source example, the logs show thousands of
RECORD
messages, but the worker still reports
Total records read: 0 (0 bytes)
. The root cause there was schema/type handling: the CSV had values that didn’t match inferred types, and the user fixed it by forcing all columns to
str
via reader options, after which rows were counted and written. [[Files source 0 records](https://discuss.airbyte.io/t/2095)] This suggests that if your low‑code connector’s output doesn’t match the declared JSON schema (wrong types, unexpected fields, etc.), the platform may drop or ignore records without surfacing a clear error. 2. Destination is unlikely to be the primary cause if “Total records read: 0” In multiple BigQuery‑related issues, when the destination is at fault you see explicit destination errors (e.g., BigQuery load failures, bad characters, platform version mismatch). [[ASCII 0 BigQuery](https://github.com/airbytehq/airbyte/issues/62134); [Platform version error](https://github.com/airbytehq/airbyte/issues/59702)] When the worker log says
Total records read: 0
, the problem is almost always on the source / message side, not BigQuery. 3. Similar “0 records but succeeded” behavior has been seen with Postgres and Files A Postgres source case shows connection issues but the sync still reports success with 0 records. [[Sync 0 records succeeded](https://discuss.airbyte.io/t/1112)] The Files source case shows records being printed but not counted until schema/type options were adjusted. [[Files source 0 records](https://discuss.airbyte.io/t/2095)] Given that, here are concrete debugging steps that are consistent with the knowledge sources: 1. Inspect the sync logs around “Total records read” - Look for lines like
Total records read: 0 (0 bytes)
and see if there are any validation or schema warnings before that (similar to the JSON schema validation messages in other threads). [[Destination BigQuery sync issues](https://discuss.airbyte.io/t/298)] 2. Verify that emitted messages are valid Airbyte RECORD messages - In the Files example, the source logs show messages like:
Copy code
json      {"type": "RECORD", "record": {"stream": "aircraft", "data": {...}, "emitted_at": 1659468274000}}
- Ensure your low‑code connector is emitting the same structure (correct
type
,
record.stream
,
record.data
,
record.emitted_at
). If the structure is off, the worker may ignore them and still report success. [[Files source 0 records](https://discuss.airbyte.io/t/2095)] 3. Check that parsed CSV fields match the declared schema types - In the Files case, mismatched types (e.g.,
NaN
where a number/string was expected) caused rows not to be ingested until everything was forced to
str
. [[Files source 0 records](https://discuss.airbyte.io/t/2095)] - For your gzipped CSV: confirm that every field you output in
record.data
is compatible with the JSON schema you defined in the low‑code spec (e.g., don’t emit
NaN
, dates as invalid strings, etc.). 4. Confirm that the stream name in records matches the configured stream - If your connector emits
stream: "X"
but the catalog/connection expects a different stream name, the platform may drop those records silently. 5. Rule out BigQuery by testing with another destination - This is a common troubleshooting step in other threads: send the same source to a simple destination like Local CSV or Postgres. If you still see
0 records
, the issue is definitely in the source / schema, not BigQuery. [[Chargebee no data](https://discuss.airbyte.io/t/2784); [Kafka 0 records](https://discuss.airbyte.io/t/901)] Because the knowledge sources don’t include low‑code–specific debugging steps, I can’t point you to an exact low‑code config fix. But based on the similar “0 records, no error” cases, the most likely causes are: - Schema/type mismatch between what your connector emits and what’s declared. - Slightly incorrect Airbyte message structure (even though your own tests show records). If you can, share a snippet of the actual
RECORD
messages from the sync logs (not just from your own test harness) and the JSON schema for that stream; that’s usually where the discrepancy shows up.
e
Did you resolve this