https://linen.dev logo
a

Ankur Adhikari

04/20/2021, 3:11 PM
Hi I am building Airbyte using " ./gradlew build " command but it gave me below error "OracleSourceTest > testReadSuccess() FAILED java.sql.SQLException: Cannot create PoolableConnectionFactory (ORA-00604: error occurred at recursive SQL level 1 ORA-01882: timezone region not found" Any solution to resolve this error ?
u

user

04/20/2021, 3:12 PM
yes, create a env called TZ=UTC
u

user

04/20/2021, 3:12 PM
export TZ=UTC
u

user

04/20/2021, 4:31 PM
we should figure out how to add this as an env variable in gradle so it's not necessary for a user to have to do this.
u

user

04/20/2021, 4:32 PM
@Ankur Adhikari i'm curious why you are running
./gradlew build
. we've tried to migrate all of our docs to recommend only building what you need. so for the core platform
CORE_ONLY=1 ./gradlew build
and for connectors building the individual connector.
u

user

04/20/2021, 4:32 PM
we suggest this because it means you run into errors that are only on the critical path for what you want to do.
u

user

04/20/2021, 4:33 PM
so if our docs are still suggesting the old thing anywhere, please let me know so I can fix it and save you and the next person some time.
u

user

04/20/2021, 4:39 PM
@charles @[DEPRECATED] Marcos Marx thanks it works for me
u

user

04/20/2021, 4:39 PM
i used CORE_ONLY=1 ./gradlew build command
u

user

04/20/2021, 4:42 PM
huh. you used the CORE_ONLY command and you got that error?
u

user

04/20/2021, 5:35 PM
yes i got that error with CORE_ONLY also until i export TZ=UTC
u

user

04/20/2021, 5:51 PM
thanks for letting me know. that shouldn't happen!
u

user

04/21/2021, 5:52 PM
created an issue to track this by the way: https://github.com/airbytehq/airbyte/issues/3010
u

user

04/21/2021, 5:53 PM
goal is to not require you to set that env variable.
u

user

04/21/2021, 5:53 PM
but i'm glad you hvae a workaround for now.
u

user

04/21/2021, 5:53 PM
@charles i want to add destination connector but when i follow steps for template generation then it only gives options to create source connector. can you please suggest if i miss anything ?
u

user

04/21/2021, 5:53 PM
nope. you're not missing anything.
u

user

04/21/2021, 5:54 PM
the path for contributing destinations is not clearly documented or templated yet.
u

user

04/21/2021, 5:54 PM
ohk
u

user

04/21/2021, 5:54 PM
long story short, there's a lot more complexity on the destination side and so we are still trying to figure out how to provide a good abstraction for contributors to use.
u

user

04/21/2021, 5:55 PM
that all being said, what destination do you want to add, I might be able to help.
u

user

04/21/2021, 5:56 PM
i want to add destination-s3 like target-s3 available in singer
u

user

04/21/2021, 5:57 PM
ah
u

user

04/21/2021, 5:57 PM
i want to say that was contributed recently actually.
u

user

04/21/2021, 5:58 PM
let me double check.
u

user

04/21/2021, 5:58 PM
yes please let us know if some one already did that
u

user

04/21/2021, 5:58 PM
thanks a lot for all support
u

user

04/21/2021, 6:00 PM
huh. i must be wrong.
u

user

04/21/2021, 6:06 PM
we're working on a related project right now and so i'm trying to figure out if a byproduct of that project will be us being able to trivially add an S3 destination. that's why i'm hesitating for another moment.
u

user

04/21/2021, 7:04 PM
we are planning to build this destination in java pretty soon (hoepfully next week or so)
u

user

04/21/2021, 7:04 PM
if you wanted to build it in java we're happy for you to just go for it.
u

user

04/21/2021, 7:05 PM
we want it in java because we have a lot of helpers that we can reuse across destinations. we also know we will want to add a lot of configurability on top of it which is why i'm being so opinionated./
u

user

04/21/2021, 7:08 PM
actually i want to use already implemented singer-based target, can you please guide me if i have to add destination then which steps i have to follow
u

user

04/21/2021, 7:10 PM
i want to implement all destinations which Rudderstack provide (https://docs.rudderstack.com/destinations)
u

user

04/21/2021, 7:57 PM
cool.
u

user

04/21/2021, 7:57 PM
you want to implement literally all of them?
u

user

04/22/2021, 6:10 AM
yes, most of them are marketing tool destinations
u

user

04/22/2021, 6:11 AM
for First Party data collection, it's really good if we collect data in our domain and then send to respective tools based on consent/compliances
u

user

04/23/2021, 8:44 AM
@charles 1. does it really mandatory to update any source/destination to Airbyte docker hub, can't we upload any source/destination to our docker hub ?
u

user

04/23/2021, 8:46 AM
2. i added a source using available script but there is no entry related to new added source in STANDARD_SOURCE_DEFINITION folder under config
u

user

04/23/2021, 4:01 PM
1. it not mandatory! we always prefer that people share their connectors so that other people in the community can use them.
u

user

04/23/2021, 4:02 PM
2. look for a source_definitions.yaml file. you'll see a list of all of the connectors in there. you can then use
./gradlew :airbyte-config:init:build
to generate the needed configuration so that it will show up in the UI. (after you make this change you'll need to recreate the docker volumes associated with aribyte)
u

user

04/23/2021, 4:02 PM
alternatively in the admin UI of airbyte you can add the image of the connector you created.
u

user

04/23/2021, 6:49 PM
Hi @charles i tried to add connector from UI also but it shows me 200 response but i am not able to see my test connector on UI. "airbyte-server | 2021-04-23 184727 INFO i.a.s.RequestLogger(filter):107 - {workspace_app_root=/tmp/workspace/server/logs} - REQ 172.18.0.1 OPTIONS 200 /api/v1/destination_definitions/create" Here are my steps: 1. ./tools/integrations/manage.sh build airbyte-integrations/connectors/destination-csv-new 2. docker tag airbyte/destination-csv-new blotout/destination-csv-new:0.2.4 3. docker push blotout/destination-csv-new:0.2.4 4. add a entry in destination yaml file 5. ./gradlew airbyte configinit:build 6. CORE_ONLY=1 ./gradlew build 7. VERSION=dev docker-compose up
u

user

04/23/2021, 10:53 PM
try running this. it will clear out the docker volumes
Copy code
VERSION=dev docker-compose down -v
VERSION=dev docker-compose up -V
u

user

04/25/2021, 11:15 AM
it works for me , thanks @charles
3 Views