<@U04S8PLJWMT> sync has a parent stream and child ...
# replication-ask-ai
b
@kapa.ai sync has a parent stream and child stream , the parent loads data into the destination internal table see "BigqueryBatchStandardInsertsLoader(finish):100 Finished loading data into table" however child stream never starts with log "destination INFO DefaultDispatcher-worker-4 i.a.c.l.s.CheckpointManager(flushReadyCheckpointMessages):148 No checkpoints to flush" continuing to show.. what could be causing this ?
k
New message text here
b
I don't see this issue in an exsiting Notion connectors forked for but setting up a new Notion connector with a new endpoint "Notion Markdown" API as the chlid stream we we see this error . is there anything in relation to this particualr stream causing an issue?
@kapa.ai I don't see this issue in an exsiting Notion connectors forked for but setting up a new Notion connector with a new endpoint "Notion Markdown" API as the chlid stream we we see this error . is there anything in relation to this particualr stream causing an issue?
k
Based on the knowledge sources provided, I can share what's documented about the Notion connector, but I need to be transparent about the limits of what's available. What the Knowledge Sources Say About Notion Connector The knowledge sources do not mention a "Notion Markdown" API or a child stream specifically for that endpoint. However, there are a few relevant points about the Notion connector's child stream behavior: 1. The
Comments
stream
(a child stream in the Notion connector) had a known breaking change when migrating from the Python CDK to the low-code framework, specifically around how incremental state is managed for partitioned/nested streams. Users were advised to clear data for that stream before resuming syncs. [Notion Migration Guide] 2. The
Blocks
stream
uses a custom
BlocksRetriever
that recursively fetches child blocks up to 30 levels deep, which can generate a very large number of API calls. [Notion components] 3. Rate limiting: The Notion API enforces ~3 requests per second per integration, and deeply nested child streams can significantly amplify the number of API calls. [Notion performance] Regarding Your Specific Issue The knowledge sources do not contain any information about a "Notion Markdown" API endpoint or a child stream built on top of it. Since you're forking the Notion connector and adding a new custom child stream, the issue you're seeing (child stream never starting,
No checkpoints to flush
repeating) is more likely related to the broader known issue discussed in our previous exchange — where
source-declarative-manifest
does not emit terminal stream status messages required by CDK v3 destinations. [streams without status] I'd recommend verifying whether your new child stream is correctly emitting stream status messages, as this is the most documented cause of the symptom you're describing. The knowledge sources don't provide enough detail to pinpoint anything specific to your custom "Notion Markdown" stream. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.
b
@kapa.ai it logs this "2026-04-20 124745 replication-orchestrator INFO MessageProcessor started. 2026-04-20 124745 replication-orchestrator INFO DestinationWriter started. 2026-04-20 124745 replication-orchestrator INFO DestinationReader started. 2026-04-20 124745 replication-orchestrator INFO SourceReader started. 2026-04-20 124745 replication-orchestrator INFO Workload successfully transitioned to running state 2026-04-20 124747 source INFO Starting syncing 2026-04-20 124747 source INFO Marking stream pages as STARTED 2026-04-20 124747 source INFO Syncing stream: pages 2026-04-20 124747 source INFO Marking stream page_markdown as STARTED 2026-04-20 124747 source INFO Syncing stream: page_markdown 2026-04-20 124747 source INFO Marking stream pages as RUNNING 2026-04-20 124747 replication-orchestrator INFO Stream status TRACE received of status: STARTED for stream pages 2026-04-20 124747 replication-orchestrator INFO Sending update for pages - null -> RUNNING 2026-04-20 124747 replication-orchestrator INFO Stream Status Update Received: pages - RUNNING 2026-04-20 124747 replication-orchestrator INFO Creating status: pages - RUNNING 2026-04-20 124747 replication-orchestrator INFO Stream status TRACE received of status: STARTED for stream page_markdown 2026-04-20 124747 replication-orchestrator INFO Sending update for page_markdown - null -> RUNNING 2026-04-20 124747 replication-orchestrator INFO Stream Status Update Received: page_markdown - RUNNING 2026-04-20 124747 replication-orchestrator INFO Creating status: page_markdown - RUNNING 2026-04-20 124747 replication-orchestrator INFO Stream status TRACE received of status: RUNNING for stream pages 2026-04-20 124910 replication-orchestrator INFO Records read: 5000 (8 MB) 2026-04-20 124934 replication-orchestrator INFO Stream status TRACE received of status: RUNNING for stream pages 2026-04-20 124934 replication-orchestrator INFO Stream status TRACE with status RUNNING received of sub-type: RATE_LIMITED for stream pages 2026-04-20 124934 replication-orchestrator INFO Sending update for pages - RUNNING -> RATE_LIMITED 2026-04-20 124934 replication-orchestrator INFO Stream Status Update Received: pages - RATE_LIMITED 2026-04-20 124934 replication-orchestrator INFO Updating status: pages - RATE_LIMITED 2026-04-20 130130 source INFO UserDefinedBackoffException: Rate limit exceeded (HTTP 429). Retrying in 714.0 seconds. 2026-04-20 130130 replication-orchestrator INFO Sending update for pages - RATE_LIMITED -> RUNNING 2026-04-20 130130 replication-orchestrator INFO Stream Status Update Received: pages - RUNNING 2026-04-20 130130 replication-orchestrator INFO Updating status: pages - RUNNING 2026-04-20 130217 replication-orchestrator INFO Records read: 10000 (19 MB)" but see how 'page_markdown' never starts after alll ared read . best apparoch here ? currenlty we just see '2026-04-20 141846 destination INFO DefaultDispatcher-worker-2 i.a.c.l.s.CheckpointManager(flushReadyCheckpointMessages):148 No checkpoints to flush' continuet to be printed "2026-04-20 141846 destination INFO DefaultDispatcher-worker-2 i.a.c.l.s.CheckpointManager(flushReadyCheckpointMessages):148 No checkpoints to flush"
k
Looking at your logs, the pattern is clear: •
pages
stream starts, runs, hits a rate limit (HTTP 429), retries, and eventually finishes reading all records. •
page_markdown
is marked as STARTED and RUNNING very early (at
12:47:47
), but never actually processes any records or emits a terminal status. • The destination then loops indefinitely with
No checkpoints to flush
. What the Logs Reveal The
page_markdown
stream being marked
RUNNING
immediately at startup — before
pages
has even finished — suggests it is a child/substream that depends on
pages
as its parent. The child stream appears to be waiting for parent records to iterate over, but something in the handoff is broken. This is directly consistent with the known bug documented in the knowledge sources: the source (using
source-declarative-manifest
) does not emit a terminal stream status message
for the parent stream (
pages
), so the destination never knows the parent is complete and cannot advance. [streams without status] Additionally, there is a known issue specifically with low-code connectors where the back-off/rate-limit handling is not correctly applied to parent streams, which can disrupt the parent→child data flow. [backoff on parent streams] Best Approaches Based on the knowledge sources, here are the documented options: 1. Downgrade your destination to a pre-CDK v3 version (e.g., BigQuery v2.x, or S3 pre-v1.0.0). CDK v3 destinations require terminal stream status messages that
source-declarative-manifest
does not emit. This is the most commonly reported workaround. [streams without status] 2. Build a custom Python source connector using the traditional Python CDK instead of the declarative manifest approach. The Python CDK correctly emits
STREAM_STATUS
messages, making it compatible with all destination versions including CDK v3. [streams without status] 3. Check the rate limit handling for the parent stream — your logs show a very long backoff (714 seconds) on the
pages
stream. There is a known bug where back-off strategies are not correctly applied to parent streams in low-code connectors, which can cause the child stream to stall. [backoff on parent streams] The knowledge sources do not confirm an official fix has been released for the
source-declarative-manifest
terminal status issue as of the latest available information, so the custom Python CDK approach is the most reliable long-term solution if downgrading the destination is not viable. 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups or DM the app for 1-to-1 help.