<#C01AHCD885S|> I see these errors on a simple ful...
# ask-ai
j
#C01AHCD885S I see these errors on a simple full refresh overwrite connection: 2025-12-03 045054,718 [io-executor-thread-4] ERROR i.a.c.s.e.h.UncaughtExceptionHandler(handle):33 - Uncaught exception java.lang.IllegalStateException: Duplicate key 54 (attempted merging values {StreamNameAndNamespace(name=core_facility_ratings, namespace=COPAY)=full_refresh, StreamNameAndNamespace(name=stage_plantype, namespace=COPAY)=full_refresh, StreamNameAndNamespace(name=core_mean_prices_combined, namespace=COPAY)=full_refresh, StreamNameAndNamespace(name=core_facility_top_problems, namespace=COPAY)=full_refresh, StreamNameAndNamespace(name=stage_facilities, namespace=COPAY)=full_refresh, StreamNameAndNamespace(name=stage_facility_types, namespace=COPAY)=full_refresh} and {StreamNameAndNamespace(name=core_facility_ratings, namespace=COPAY)=full_refresh, StreamNameAndNamespace(name=stage_plantype, namespace=COPAY)=full_refresh, StreamNameAndNamespace(name=core_mean_prices_combined, namespace=COPAY)=full_refresh, StreamNameAndNamespace(name=core_facility_top_problems, namespace=COPAY)=full_refresh, StreamNameAndNamespace(name=stage_facilities, namespace=COPAY)=full_refresh, StreamNameAndNamespace(name=stage_facility_types, namespace=COPAY)=full_refresh}) at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:135) at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:182) at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) at io.airbyte.commons.server.handlers.helpers.StatsAggregationHelper.hydrateWithStats(StatsAggregationHelper.kt:181) at io.airbyte.commons.server.handlers.helpers.StatsAggregationHelper.getJobIdToJobWithAttemptsReadMap(StatsAggregationHelper.kt:306) at io.airbyte.commons.server.handlers.ConnectionsHandler.getConnectionLastJobPerStream(ConnectionsHandler.kt:2188) at io.airbyte.server.apis.controllers.ConnectionApiController.getConnectionLastJobPerStream$lambda$18(ConnectionApiController.kt:281) at io.airbyte.server.apis.ApiHelper.execute(ApiHelper.kt:32) at io.airbyte.server.apis.controllers.ConnectionApiController.getConnectionLastJobPerStream(ConnectionApiController.kt:280) at io.airbyte.server.apis.controllers.$ConnectionApiController$Definition$Exec.dispatch(Unknown Source) at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invokeUnsafe(AbstractExecutableMethodsDefinition.java:461) at io.micronaut.context.DefaultBeanContext$BeanContextUnsafeExecutionHandle.invokeUnsafe(DefaultBeanContext.java:4438) at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:272) at io.micronaut.web.router.DefaultUriRouteMatch.execute(DefaultUriRouteMatch.java:38) at io.micronaut.http.server.RouteExecutor.executeRouteAndConvertBody(RouteExecutor.java:465) at io.micronaut.http.server.RouteExecutor.lambda$callRoute$5(RouteExecutor.java:442) at io.micronaut.core.execution.ExecutionFlow.lambda$async$0(ExecutionFlow.java:92) at io.micronaut.core.propagation.PropagatedContext.lambda$wrap$3(PropagatedContext.java:232) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583)
k
This stack trace is a "duplicate key" error thrown when Airbyte is aggregating per‑stream stats for a connection (
StatsAggregationHelper.hydrateWithStats
/
getConnectionLastJobPerStream
). It’s the same type of problem as other duplicate‑key issues we’ve seen (e.g., duplicate keys in DB constraints or config maps), but there is no specific mention of this exact
StatsAggregationHelper
error pattern in the available knowledge sources. What we do know from the docs/issues you shared: - Airbyte has several duplicate‑key style bugs reported in 1.8.x, e.g. the
pk_connector_rollout
constraint violation during the cron
DefinitionsUpdater
job, which is an open issue in 1.8.0/1.8.1 and requires a product fix rather than configuration changes.[[1.8.0/1.8.1 rollout](https://github.com/airbytehq/airbyte/issues/65134)] - Duplicate‑key situations can also surface at the configuration/metadata level (e.g. Helm chart ConfigMap duplicate keys, “duplicate key” errors for actor catalog fetch events), and the recommended resolution in those cases has been to upgrade once a fix is released, or to adjust the problematic configuration.[[Helm ConfigMap dup](https://github.com/airbytehq/airbyte/issues/66546); [Actor catalog dup](https://discuss.airbyte.io/t/3750)] However, none of the sources describe: - A known bug specifically in
StatsAggregationHelper.hydrateWithStats
or
getConnectionLastJobPerStream
, or - A documented workaround for this exact
Duplicate key 54
error on full‑refresh‑overwrite jobs. Because of that, I cannot confidently tell you: - Whether this is already tracked as a specific GitHub issue, or - Whether a particular Airbyte version definitively fixes it. Given the constraints of the available information, the best I can suggest (without guessing) is: 1. Treat this as a likely product bug similar to the other duplicate‑key cases. 2. Check whether you’re on an older minor/patch version (e.g., around 1.8.x) and, if feasible, upgrade to the latest stable Airbyte version, since recent duplicate‑key problems have been addressed via upgrades in the past.[[1.8.0/1.8.1 rollout](https://github.com/airbytehq/airbyte/issues/65134)] 3. If the error persists on the latest version, open or update a GitHub issue including: - This full stack trace - Airbyte version and deployment type - Connection ID and job IDs involved - Confirmation that it occurs on a simple full‑refresh‑overwrite connection I’m sorry I can’t be more specific; the current knowledge sources simply do not describe this exact failure mode in enough detail to give a targeted fix.
j
what version of airbyte am I running? Here is what i see when I run abctl local status
INFO Using Kubernetes provider: Provider: kind Kubeconfig: /home/ubuntu/.airbyte/abctl/abctl.kubeconfig Context: kind-airbyte-abctl SUCCESS Found Docker installation: version 29.0.2 SUCCESS Existing cluster ‘airbyte-abctl’ found INFO Found helm chart ‘airbyte-abctl’ Status: failed Chart Version: 2.0.19 App Version: 2.0.1 INFO Found helm chart ‘ingress-nginx’ Status: deployed Chart Version: 4.14.0 App Version: 1.14.0 INFO Airbyte should be accessible via http://localhost:8000