I am trying to see the actual query being run during increme Airbyte #contributing-to-airbyte

I am trying to see the actual query being run duri...

Ameya Bapat

02/03/2022, 7:39 AM

I am trying to see the actual query being run during incremental sync. Added logs in after

airbyte-integrations/connectors/source-jdbc/src/main/java/io/airbyte/integrations/source/jdbc/AbstractJdbcSource.java:265

LOGGER.info("Actual query : {}", sql); I ran

./gradlew :airbyte-integrations:connectors:source-snowflake:build

and

./gradlew :airbyte-integrations:connectors:source-jdbc:build

and

VERSION=dev docker-compose up

but it is not showing my Looger.info() statements in job's log. I can see the previous and later logger statements but mine. Am i missing any step?

Ameya Bapat

02/03/2022, 2:35 PM

Hi @Ameya Bapat are you able to get the existing log line

Executing query for table:

Ameya Bapat

02/03/2022, 5:45 PM

Couple of things on my mind: 1. Make sure when you start up airbyte and the source-snowflake container pulls, its pulling your local build with your changes and not the airbyte one from airbyte's Dockerhub 2. Are you running a sync? Wouldnt that log line only show up when a sync attempts?

Arian Giles Garcia

02/04/2022, 6:59 AM

@Augustin Lafanechere (Airbyte) yes, I can see that line as it was already there and not added by me.

Arian Giles Garcia

02/04/2022, 7:00 AM

@Noah Kawasaki yes, those logs are supposed to show up only during syncs.

Arian Giles Garcia

02/04/2022, 7:02 AM

@Noah Kawasaki yes, actually it is creating docker image when i run

version=dev docker compose up

. From UI settings I was still pointing to some 0.1.6 and not to dev image. I changed that to dev. Lets see whether logs shows up or not.

Arian Giles Garcia

02/04/2022, 3:41 PM

@Ameya Bapat you can also run the read operation manually with

docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-snowflake:dev read --config /secrets/configs.json --catalog /path/to/configured_catalog.json

4 Views

Open in Slack

Previous Next