https://linen.dev logo
m

Murilo Nigris

01/13/2021, 2:27 PM
Hi Airbyte team! Do I need to submit a documentation for the source I'm developing similar to https://docs.airbyte.io/integrations/sources/braintree ? If yes, how/where do I do it?
u

user

01/13/2021, 2:28 PM
In the github repository, that page is here: https://github.com/airbytehq/airbyte/blob/master/docs/integrations/sources/braintree.md so you are welcome to add your own doc there too
u

user

01/13/2021, 2:39 PM
Awesome! Thank you @Chris (deprecated profile)!
u

user

01/13/2021, 2:39 PM
I'm finishing up this connector: https://github.com/airbytehq/airbyte/issues/1489 Do I need to create an issue for you guys to create your own credentials for Appstore API and add it to Github secrets? Or should I skip that?
u

user

01/13/2021, 2:40 PM
you can create the issue, that would help! thanks
u

user

01/13/2021, 2:40 PM
Ok!! I'll do both 🙂
u

user

01/13/2021, 2:59 PM
Done, I was not sure if I used the correct issue tags, but I put it under 'new integration' https://github.com/airbytehq/airbyte/issues/1631
u

user

01/13/2021, 4:07 PM
Thanks! We will have someone try to get those creds today. you don't have to wait to put up a PR for us to do this. 😄
u

user

01/13/2021, 5:02 PM
Great! Thanks @charles 🙂
u

user

01/13/2021, 5:03 PM
Do I do the PR with this instructions, or do I ingore them for now? 1. Once the config is stored in Github Secrets, edit 
.github/workflows/gradle.yaml
 to inject the config into the build environment. 2. Edit the 
airbyte/tools/bin/ci_credentials.sh
 script to pull the script from the build environment and write it to 
secrets/config.json
 during the build.\\
u

user

01/13/2021, 5:14 PM
you can ignore them. we will handle it once we get the credentials.
u

user

01/13/2021, 5:14 PM
Ok 🙂
u

user

01/13/2021, 5:14 PM
thanks for mentioning it. i think i will remove those from the checklist so it's not confusing for people in the future.
u

user

01/13/2021, 5:53 PM
Pull request sent. Any issues please let me know and I'll try to fix it.
u

user

01/13/2021, 6:43 PM
Thank you @Murilo Nigris airbyte heart
u

user

01/13/2021, 6:59 PM
Just reviewed it. I have only one question that I want to figure out and then we will merge.
u

user

01/13/2021, 7:00 PM
Happy to follow up on github or in this thread. 🎉
u

user

01/13/2021, 7:29 PM
@charles, we can talk here
u

user

01/13/2021, 7:29 PM
Please, go ahead
u

user

01/13/2021, 7:35 PM
u

user

01/13/2021, 7:35 PM
Should I change here and do another PR?
u

user

01/13/2021, 7:37 PM
1. Quick question about github 😅, should I commit the changes you suggested on the other files via the github UI button and then I rebase my own repo, or is there a better way?
u

user

01/13/2021, 7:46 PM
2. About your comment of checking the status_code, as the singer tap is using this library, it either returns a 'tsv' format string or will return an API error as string. In the tap code the only way we thought about coding it was:
Copy code
try:
                rep_tsv = api.download_sales_and_trends_reports(filters=
                    report_filters)
            except APIError as e:
                LOGGER.error(e)
                break
            
            # parse api response
            if isinstance(rep_tsv, dict):
                LOGGER.warning("Received a JSON response instead of the report: %s", str(rep_tsv))
                break
            else:
                rep = tsv_to_list(rep_tsv)
u

user

01/13/2021, 8:02 PM
On a second thought, I'm not sure it's ideal to change this, the alternative would be bypass the use of this library, but then it generates other problems such as incompatibility with the rest of the code. Makes sense?
u

user

01/13/2021, 8:05 PM
3. Before you approve the PR, I just tried to run the 'docker-compose up' but I'm not seeing any sources available in the UI anymore:
u

user

01/13/2021, 8:05 PM
(sorry the amount of comments)
u

user

01/13/2021, 8:48 PM
Quick question about github 😅, should I commit the changes you suggested on the other files via the github UI button and then I rebase my own repo, or is there a better way?
i would just hit commit in the ui and then git pull
u

user

01/13/2021, 8:48 PM
that should pull down the changes.
u

user

01/13/2021, 8:48 PM
if you have already made changes on your local branch then yeah you'll need to rebate.
u

user

01/13/2021, 8:49 PM
On a second thought, I'm not sure it's ideal to change this, the alternative would be bypass the use of this library, but then it generates other problems such as incompatibility with the rest of the code. Makes sense?
got it! i see that makes sense we can leave it as is.
u

user

01/13/2021, 8:51 PM
Before you approve the PR, I just tried to run the 'docker-compose up' but I'm not seeing any sources available in the UI anymore:
weird. can you run
docker-compose down
u

user

01/13/2021, 8:51 PM
then run
docker volume rm $(docker volume ls -q | grep airbyte)
u

user

01/13/2021, 8:51 PM
this will clear out the volumes
u

user

01/13/2021, 8:51 PM
and then
docker-compose up
u

user

01/13/2021, 8:51 PM
let me know if that helps!
u

user

01/14/2021, 6:18 AM
It solved the error, but the Integration is not appearing yet 😞
u

user

01/14/2021, 6:19 AM
DId you test in your machine the PR? Did it work for you?
u

user

01/14/2021, 7:25 AM
oh! I see. so until we merge it, it can only be accessed in "dev mode"
u

user

01/14/2021, 7:25 AM
you can see the instructions for how to run airbyte in dev mode in our docs here: https://docs.airbyte.io/contributing-to-airbyte/developing-locally#run-in-dev-mode-with-docker-compose
u

user

01/14/2021, 7:25 AM
lmk if any how to get that running is unclear.
u

user

01/14/2021, 1:27 PM
I'll try it 🙂
u

user

01/14/2021, 1:28 PM
Is still missing anything from my part regarding the PR? Or now it's with you guys?
u

user

01/14/2021, 3:58 PM
Charles, the cmd in the URL above returns only this:
Copy code
./gradlew build
docker-compose --env-file .env.dev -f docker-compose.yaml -f docker-compose.dev.yaml up
u

user

01/14/2021, 4:30 PM
weird!
u

user

01/14/2021, 4:30 PM
i just merged your PR. appstore will be visible once i bump the airbyte version. will ping you when that's done.
u

user

01/14/2021, 4:50 PM
Awesome!! Thank you 😄
u

user

01/14/2021, 4:51 PM
it'll probably take me about an hour.
u

user

01/14/2021, 4:51 PM
Ok, I'm starting to work on the appsflyer one: https://github.com/airbytehq/airbyte/issues/1649
u

user

01/14/2021, 4:52 PM
I hope this is ok
u

user

01/14/2021, 4:52 PM
yeah! that's great. are you going to try to do it without singer?
u

user

01/14/2021, 4:52 PM
would be curious if you found that easier.
u

user

01/14/2021, 4:52 PM
Not really, I'm under a tight deadline. The next after that I can try 🙂
u

user

01/14/2021, 4:52 PM
ok
u

user

01/14/2021, 4:52 PM
no problem.
u

user

01/14/2021, 4:53 PM
Which I hope I also start today hehe
u

user

01/14/2021, 5:12 PM
remind me what destination you're using again?
u

user

01/14/2021, 5:25 PM
BigQuery
u

user

01/14/2021, 5:26 PM
Why?
u

user

01/14/2021, 5:26 PM
just curious. i'm interested to know which sources and destinations people are using.
u

user

01/15/2021, 12:29 AM
you should be able to see your source in the app now on master.
u

user

01/15/2021, 1:03 PM
Hi @charles! I'm not seeing the appstore connector. I was checking the commits and saw that the CI run failed due to the lack of docker image: https://github.com/airbytehq/airbyte/runs/1703212511
u

user

01/15/2021, 7:51 PM
weird. i'll take a look this afternoon.
u

user

01/15/2021, 7:59 PM
Thankss!!
u

user

01/15/2021, 8:19 PM
I see it in the dropdown.
u

user

01/15/2021, 8:20 PM
can you turn off airbyte
docker-compose down -v
. git pull (to get newest version of airbyte) and then
docker-compose up -V
3 Views