<@U04S8PLJWMT> please provide the documentation fo...
# replication-ask-ai
d
@kapa.ai please provide the documentation for custom built connectors
k
Here's an overview of the documentation for custom-built connectors in Airbyte: Building Custom Connectors When to Build a Custom Connector In most cases, you should use the Connector Builder for API source connectors that support HTTP requests (REST, GraphQL) and return JSON/JSONL formats. You should only build a fully custom connector when the Builder does not support your data source or destination. [Custom or New Connector] Options for Building Custom Connectors 1. Connector Builder – The recommended starting point for most API connectors. [Connector Development] 2. Low-Code CDK – For more complex cases where the Builder isn't sufficient. [Connector Development] 3. Python CDK or Java CDK – For advanced use cases requiring full programmatic control. [Custom or New Connector] 4. Any language via Docker – Airbyte connectors can technically be written in any language, deployed as custom Docker images. [Custom or New Connector] Custom Components in Connector Builder If you're using the Connector Builder but need to extend it with custom logic, Custom Components allow you to write Python classes that implement specific Airbyte CDK interfaces. They follow this pattern: • A dataclass implementing the interface of the component it replaces • Fields representing configurable arguments from the YAML configuration • Implementation of required methods for the component's capability Use cases include: • Unusual pagination patterns or complex authentication • Specialized data transformations • Edge cases not covered by built-in components [Custom Components] > ⚠️ Custom Components are available on Enterprise Flex and Self-Managed Enterprise plans only. Process Overview 1. Pick the technology and build your connector (Builder, Low-Code CDK, Python CDK, etc.) 2. Publish as a custom connector to make it available in your workspace 3. Contribute back to Airbyte (optional) – submit it to the connector Marketplace via the contribution guide [Connector Development] Contributing a Custom Python/Java CDK Connector If you want to contribute a Python CDK or Java CDK connector to Airbyte's catalog: 1. Check for an existing GitHub Discussion or open a new one 2. Ensure your connector passes the test suite 3. Include a README, documentation, and an icon 4. Follow the pull request title convention 5. Cover authentication, pagination, and incremental syncs where applicable 6. Provide a sandbox account if requested [Submit New Connector] Additional Resources • Connector Development Guide • Custom Python Connector Tutorial • Airbyte Protocol Specification • #C027KKE4BCZ 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
d
@kapa.ai having issues with the amazon connector, it only syncs the orders stream, I need inventory as well but it always returns a 403 - even though the app on seller central has full permissions
k
From the available information, 403 errors on Amazon Seller Partner streams (including inventory- and report-type streams) are most often caused by missing or insufficient SP‑API permissions for that specific endpoint/report type, even when the app appears to have “full” permissions in Seller Central. What we know from the knowledge sources: • Several streams (including various reports) are known to fail with
403 Forbidden
when the account or sandbox doesn’t have the right access; these are explicitly documented as permission issues, not connector bugs. [Known stream failures] • In other support cases, a
403 Client Error: Forbidden for url: https://sellingpartnerapi-.../reports/2021-06-30/reports
was traced back to insufficient Amazon permissions, and users were advised to verify access with Amazon. [Forum normalization thread] • Another forum thread shows
Access to the resource is forbidden
(
SellingApiForbiddenException
) when trying to create reports; again this was treated as a permissions / developer profile problem, not an Airbyte bug. [Permissions: SellingApiForbiddenException] • The official connector docs emphasize that certain data (e.g., Brand Analytics, PII in Orders/OrderItems) requires specific roles or approvals in the SP‑API developer profile beyond just having an app authorized. [Amazon SP source docs] The knowledge sources do not list a specific, known bug for inventory streams returning 403 (unlike the settlement report bug that was fixed by correcting a URL path). [Settlement report 403 bug] So based on what we have, your issue is most likely permissions / access-scope related, not a confirmed connector defect. Given that, here are the concrete steps you can take (all consistent with the docs and forum guidance): 1. Confirm which exact inventory stream is failing For example, some inventory-related data comes from report streams (e.g., FBA inventory reports) rather than a simple REST endpoint. Many of those report streams are known to 403 or time out in certain environments if the account doesn’t have the right access. [Known stream failures] 2. Verify SP‑API roles and report access with Amazon ◦ Check in Seller Central / Developer Console that your SP‑API application has: ▪︎ The correct role(s) for the specific inventory/report APIs you’re calling. ▪︎ Any required Brand Registry / Brand Analytics or other specialized roles if applicable. [Amazon SP source docs] ◦ If possible, call the same inventory/report endpoint directly (e.g., via curl or Postman) using the same credentials. In other issues, this was the recommended way to confirm whether the 403 is from Amazon rather than Airbyte. [Amazon Seller Partner connection error] 1. Re‑authenticate / recreate the source after permission changes When permissions or roles are updated in Amazon, you may need to: ◦ Re‑authorize the app in Seller Central. ◦ Edit the Airbyte source and go through authentication again so the connector uses the updated scopes. [Amazon SP setup step 2] 1. Check if only some streams fail It’s expected that: ◦ Some streams (like
Orders
) work. ◦ Others (inventory/report streams) 403 if your account doesn’t have access to those specific APIs or report types. This pattern appears in multiple forum threads and in the “known failures” list. [Known stream failures] Because the knowledge sources don’t include your exact logs or connector version, I can’t definitively say whether there’s a bug in your specific inventory stream. From what is documented, though, the next best step is: • Verify with Amazon that your SP‑API app is explicitly allowed to access the exact inventory/report APIs you’re trying to sync. • After confirming that, re‑authenticate the Airbyte source and retry. If you can share: • The exact stream name(s) that 403, and • The full 403 error message / URL path from the Airbyte logs, I can cross‑check them against the documented known issues and bugs in the Amazon Seller Partner connector. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.